How to inject API server URL when deploying react frontend?

You can try this:

// http.js
const getBaseUrl = () => {
  let url;
  switch(process.env.NODE_ENV) {
    case 'production':
      url = 'https://stackoverflow.com';
      break;
    case 'development':
    default:
      url = 'https://google.com';
  }

  return url;
}

export default axios.create({
  baseURL: getBaseUrl(),
});

Using this package https://github.com/toddbluhm/env-cmd you could create an env file for your environment

for example create .env.staging and .env file with this code

// .env.staging file   
API_URL=https://staging.url.com/api/

// .env file
API_URL=https://url.com/api/

How to fetch with API_URL from env variable:

fetch(process.env.API_URL)

Then you can just add some extra scripts in your package.json:

{
  "scripts": {
    "build:staging": "env-cmd .env.staging yarn build",
    "build:prod": "env-cmd .env yarn build"
  }
}