OK so this really freaked me out! Help?

Spirits can be attached to ANYthing, could be the land that the house is on. or a peice of furniture. You can go to the library and or the court house and find out if they're was a home there before yours, you can also research the land, could be an old cematery, or a battle ground. not sure where you are, but you can do that or call ghost hunters, get a preist to come over and bless the home. but if they aren't physically hurting you I don't think I would worry about it.