This common misconception may stem from the idea that electrons are being “stored in the battery”, meaning that the battery’s charge would have some equivalent mass. Also, when I was a child, my dad told me that heavier batteries (like single-use AA alkaline batteries) were better than their lighter counterparts. In my mind I interpreted that to mean that a heavier battery had been used less than a lighter weight one and had more charge remaining.
Really what he meant is that heavier batteries probably have a greater chemical mass by design and therefore were manufactured with a higher initial charge, and therefore heavier meant higher quality and longer lasting. Sometime compare the no-name AA battery’s mass that came with your TV remote to a name brand AA battery. You’ll probably see a difference.
With all of that said, the short answer is: “no, the mass doesn’t change”
The total number of electrons doesn’t change in a battery when it is charged or discharged. The electrons simply move from one side of the reaction to the other (a high state to a low state when discharging or vice-versa for charging). Since the quanitity of electrons does not change in the battery, the mass does not change.
However, it should be noted that the density of some batteries will change as they charge and discharge. For example many lithium-ion cells (especially pouch-style) will inflate slightly as they charge. This leads to a reduction in the battery’s density since density is mass / volume.
To this point, this answer refers to typical battery technologies
However, some experimental and developing battery technologies use their environment as part of the chemical reaction. One promising technology is the Lithium-Air battery which actually uses the surrounding oxygen in the air. In this type of battery, the mass of the physical battery structure actually does increase as the electrolyte takes on oxygen from the atmosphere.
A similar decrease in mass occurs as the oxygen is later expelled. This example is a bit unfair though since technically speaking the oxygen that was used and released was indeed part of the battery and therefore the entire system including that oxygen has the same mass at all times regardless of charge state.