This has been previously discussed; see this post for a summary, and perhaps take a look at the whole thread. A quick search for “annotation file size” reveals that and other relevant threads too.
Which version of macOS do you use and what kind of annotation? I just tried this on macOS 11.4 using various annotations but the PDF document remains editable after saving, closing & reopening.
The file size is increased, quite likely due to a difference in compression algorithms used and PDFKit, but I’m not seeing an issue highligthing that PDF.