About SVG
SVG is a text format and in most cases it will be compressed before network transfer, so do any comparision after gzip (see above).
I advice to choose SVG in most cases bacause of its vector nature. It scales well on any device.
You can insert SVG in DOM, as an image using IMG or place it on background in CSS. When an image is small and is used as decoration it is better to place it in CSS background property as data:uri (https://en.wikipedia.org/wiki/Data_URI_scheme) using URLEncode method (https://en.wikipedia.org/wiki/Percent-encoding).
I use SVGO (https://github.com/svg/svgo) to optimize my images with such parameters:
svgo -p 2 --multipass --enable=removeDesc --enable=removeTitle --enable=sortAttrs --enable=removeViewBox --enable=removeStyleElement
Typically it is enough.
But if you want to fine-tune SVG image and achieve minimal image size you need "to go deeper". In some cases it is better to "draw" an image in text editor instead of visual one. So I suggest to learn SVG internals, e.g. using this tutorial:
https://developer.mozilla.org/en-US/docs/Web/SVG/TutorialAnd one more rarery used thing about SVG optimization. Usually SVG has the same dimentions as will be used on web page. In such case vector coordinates may be fractional and it can increase file size. You may scale image to place vector points on pixel grid.
Example:
https://yastatic.net/web4/_/BM4YHAtH2zfvUiPvyhZHMkaXw4w.svgBTW, you may find an example of xlink usage there but that's not always the best way. Sometimes gzip works better than symbol.