Context Deadline Exceeded - prometheus

I had a similar problem, so I tried to extend my scrape_timeout but it didn't do anything - using promtool, however, explained the problem

My problematic job looked like this:

- job_name: 'slow_fella'
  scrape_interval: 10s
  scrape_timeout: 90s
  static_configs:
  - targets: ['192.168.1.152:9100']
    labels:
      alias: sloooow    

check your config like this:

/etc/prometheus $ promtool check config prometheus.yml

Result explains the problem and indicates how to solve it:

Checking prometheus.yml
  FAILED: parsing YAML file prometheus.yml: scrape timeout greater than scrape interval for scrape config with job name "slow_fella"

Just ensure that your scrape_timeout is long enough to accommodate your required scrape_interval.


Probably the default scrape_timeout value is too short for you

[ scrape_timeout: <duration> | default = 10s ]

Set a bigger value for scrape_timeout.

scrape_configs:
  - job_name: 'prometheus'

    scrape_interval: 5m
    scrape_timeout: 1m

Take a look here https://github.com/prometheus/prometheus/issues/1438


I had a same problem in the past. In my case the problem was with the certificates and I fixed it with adding:

 tls_config:
      insecure_skip_verify: true

You can try it, maybe it will work.