Parasitic draw can definitely do it. You can easily test by disconnecting everything from the battery except the charger. Bring it to full charge, and then check it about 12 hours after charging, and then daily for 5 or 6 days all the while with nothing connected. Easy by using voltmeter readings. If it holds charge, your problem is verified and will be resolved by adding a disconnect as suggested. Some other possibilities are battery, charger, charging habits, wiring or connection issue, or electrolyte level in flooded battery.