How to write a PyTorch sequential model?

Sequential does not have an add method at the moment, though there is some debate about adding this functionality.

As you can read in the documentation nn.Sequential takes as argument the layers separeted as sequence of arguments or an OrderedDict.

If you have a model with lots of layers, you can create a list first and then use the * operator to expand the list into positional arguments, like this:

layers = []
layers.append(nn.Linear(3, 4))
layers.append(nn.Sigmoid())
layers.append(nn.Linear(4, 1))
layers.append(nn.Sigmoid())

net = nn.Sequential(*layers)

This will result in a similar structure of your code, as adding directly.

As described by the correct answer, this is what it would look as a sequence of arguments:

device = torch.device('cpu')
if torch.cuda.is_available():
    device = torch.device('cuda')

net = nn.Sequential(
      nn.Linear(3, 4),
      nn.Sigmoid(),
      nn.Linear(4, 1),
      nn.Sigmoid()
      ).to(device)


print(net)

Sequential(
  (0): Linear(in_features=3, out_features=4, bias=True)
  (1): Sigmoid()
  (2): Linear(in_features=4, out_features=1, bias=True)
  (3): Sigmoid()
  )

As McLawrence said nn.Sequential doesn't have the add method. I think maybe the codes in which you found the using of add could have lines that modified the torch.nn.Module.add to a function like this:

def add_module(self,module):
    self.add_module(str(len(self) + 1 ), module)

torch.nn.Module.add = add_module

after doing this, you can add a torch.nn.Module to a Sequential like you posted in the question.