Is it OK to add citations from low impact factor journals?

Of course, you should cite any relevant work.

You can't ignore work just because it is in a journal that someone believes to be lesser. That doesn't mean the work is bad.

As long you do a thorough literature review and avoid predatory journals, there should be no negative impact on your paper.

The impact factor (IF) is a metric of citations. If enough manuscripts from a low-IF journal get cited more often, the journal's IF will improve.

In general, you should consider any manuscript independent of the journal it got published in. However, there is one caveat: If you suspect that a journal doesn't follow a good review process (such as predatory journals), you should examine the manuscript with extra caution. Furthermore, there is usually some correlation between the quality of a manuscript and the work described therein and the quality of the journal it got published in.

As a reviewer, I have no problem with a few citations from low-IF journals. However, I will be very critical if there are no citations from mid- to high-IF journals because that is an indication that your work might not be very relevant or innovative.

Your own work is not based on the impact factor of your cited work. The quality of your argumentation and thesis you discuss in a paper mainly builds on a coherent discussion of the state of research, independent from where it is published. Only referencing high-impact factor journals indicates that you only did a shallow literature search in the more well-known journals, which is not very professional in my opinion.