Discrepancy between two hosts running the same docker commands
How do we fix it?
Your error reports a GCC / compilation problem.
A quick search shows mostly problems related to python / gcc version (one, two, three).
But you are right, this doesn't look like as it could happen inside a one particular container.
What it does look like is some kind of OOM problem.
Also, is this a VM? Stan requires a significant amount of memory to compile the models, and this error can occur if you run out of RAM while it is compiling.
I did a bit of testing.
On my machine the compilation process consumed up to 2.4 Gb of RAM.
cat /etc/redhat-release
CentOS Linux release 7.9.2009 (Core)
uname -r
3.10.0-1160.6.1.el7.x86_64
docker --version
Docker version 20.10.1, build 831ebea
# works fine
docker run --rm -it -m 3G python:3.8.6 /bin/bash
# fails with error: command 'gcc' failed with exit status 1
# actually it was killed by OOM killer
docker run --rm -it -m 2G python:3.8.6 /bin/bash
# yes, here he is
tail -f /var/log/messages | grep -i 'killed process'
Dec 22 08:34:09 cent7-1 kernel: Killed process 5631 (cc1plus), UID 0, total-vm:2073600kB, anon-rss:1962404kB, file-rss:15332kB, shmem-rss:0kB
Dec 22 08:35:56 cent7-1 kernel: Killed process 5640 (cc1plus), UID 0, total-vm:2056816kB, anon-rss:1947392kB, file-rss:15308kB, shmem-rss:0kB
Check OOM killer log on problematic machine.
Is there enough RAM available for Docker?
Can upgrading the docker engine version break a container?
Generally, it shouldn't be the case.
But for v20.10.0 Docker introduced a very big set of changes related to memory and cgroups.
After you rule out all obvious reasons (like your friend's machine just not having enough RAM), you might need to dig into your docker daemon settings related to memory / cgroups / etc.
How can the same container produce different results on two computers?
Well, technically it's quite possible.
Containerized programs still use host OS kernel.
Not all kernel settings are "namespaced", i. e. can be set exclusively for one particular container.
A lot of them (actually, most) are still global and can affect your program's behavior.
Though I don't think it's related to your problem.
But for complicated programs relying on specific kernel setting that must be taken into account.