How to avoid keeping version number in source code?

Premises:

I assume these are the premises under which the solution is discussed.

1. Currently version number is kept in a git-tracked source file, but you are OK to get rid of it.
2. No one manually manages version number, nor triggers a release procedure, which includes: (a) increase version number, (b) build from source and (c) store the built result somewhere. These are taken cared by CI, and SHOULD remain that way.

Solution:

1. Instead of writing to a source file and create new commit, CI simply tag the specific commit that passed CI check, then push the tag to remote repo.
2. The build script should read the tag of current HEAD commit, and use it as the version number for publishing a release.
3. Optionally, you might want to use git filter-branch to rewrite your existing git repo history, tag previous release commits for consistency, remove and stop tracking the version number source cile, then get rid of those CI commits.

It is a common practice to keep a version number in the source code, there is nothing wrong in that.

You need to separate CI procedures to regular builds, release publishing and release deployment.

Regular builds: run daily or even after each commit, can include static code analysis and automatic tests, check if the code can be built at all. Regular builds should not change the version number.

Release publishing: can only be triggered by explicit manual action by release manager.
The trigger action could be tagging a commit with a new version number, new merge into the release branch, or just a commit that changes version number kept in a special file (e.g. pom.xml). Refer to git flow for example.
Release publishing assigns a new version number (either automatically or manually), commits it into the source code if necessary, builds a binary package with a new version and uploads it to the binary package repository (e.g. Nexus, devpi, local APT repository, Docker registry and so on).

Release deployment: another manually triggered action that takes a ready binary package from a package repository and installs it to the target environment (dev, QA / UAT / staging, part of production for canary deployments or to the whole production environment).