I had many of the same side effects. Actually just about everything you listed accept the joint pain but I did have backaches, migraines (which I had never had in my life until I had the Mirena inserted,) acne, lightheadedness, the funny heart feeling, depression, moodiness, fatigue, and others. I did gain about 10 pounds since insertion and still have not lost it. I had my Mirena for 3 1/2 years and had it removed about 3 months ago.
I can tell you, that for me, many of the symptoms left right away. The funny heart thing/panic attach (it is difficult to describe, I would be just sitting and watching TV and my heart felt like it was not beating right and I felt "off.") Well this is completely gone, the backaches stopped immediately. The Acne took about 2 months to go away. I did have times where it seemed some symptoms flared up from time to time like the Depression but it is getting less and less over time. I think it takes some time for the hormone to work out of the body.
I don't doubt that most of what you felt was caused by the Mirena. Now that it is gone, your symptoms should go away but it may take a little time. It does affect everyone differently though.
Hope this helps.