What does data.norm() < 1000 do in PyTorch?
Building on what @kmario23 says, the code multiplies the elements of a vector by 2 until the Euclidean magnitude (distance from origin) / L2 norm of the vector is at least 1000.
With the example of the vector with (1,1,1): it increases to (512, 512, 512), where the l2 norm is about 886. This is less than 1000, so it gets multiplied by 2 again and becomes (1024, 1024, 1024). This has a magnitude greater than 1000, so it stops.
It's simply the L2 norm (a.k.a Euclidean norm) of the tensor. Below is a reproducible illustration:
In [15]: x = torch.randn(3, requires_grad=True)
In [16]: y = x * 2
In [17]: y.data
Out[17]: tensor([-1.2510, -0.6302, 1.2898])
In [18]: y.data.norm()
Out[18]: tensor(1.9041)
# computing the norm using elementary operations
In [19]: torch.sqrt(torch.sum(torch.pow(y, 2)))
Out[19]: tensor(1.9041)
Explanation: First, it takes a square of every element in the input tensor x
, then it sums them together, and finally it takes a square root of the resulting sum. All in all, these operations compute the so-called L2 or Euclidean norm.
y.data.norm()
is equivalent to
torch.sqrt(torch.sum(torch.pow(y, 2)))