Do SAS expanders work transparently with SAS controllers?
First, stop and think about your realistic goals...
The short answer to your question is that SAS is a switching protocol. The drives connected via expanders are all accessible and can be addressed transparently by a single controller.
However, the rest of your questions are going to be a function of your particular controller, your storage architecture and the backplanes in the storage enclosure(s) you end up using.
It's not possible to give a specific answer without more information.
- How many disks do you need?
- What RAID controller or SAS HBA are you using?
- Will these disks be internal or external?
- What specific SAS expander card are you looking to use (it's not like there are many on the market)?
- What server and enclosure hardware are you considering?
- What is the application/reason for this setup? (ZFS? Storage Spaces?)
Let's use your example of a controller with 4 x SAS SFF-8087 ports. These are meant for communication inside of a chassis. Each SFF-8087 port is a 4-lane connector that can accommodate 4 drives. 16 drives total, in your case... Servers typically only have 4, 8, 12, 16, or 24/25 internal drive bay capacities.
At the 4 and 8-drive level, the internal server drive backplane will have a 1:1 lane-to-drive ratio and typically have one or two SFF-8087 ports. No oversubscription!
For 12-port servers, the expander is usually built into the backplane, and the input will be a single SFF-8087 port. Oversubscription!
For servers with 16 internal bays, you may see four separate SFF-8087 ports intended to be served by a single or multiple SAS/RAID controllers. But at 16-ports, you may have an expander on the backplane instead.
Almost every 24 or 25-bay server I've encountered has expanders built onto the drive backplane and accommodate one or two SFF-8087 ports. The exception is something like the HP ProLiant DL370 G6, which could accept three 8-bay drive cages, each requiring 2 SFF-8087 ports.
So when do people use PCIe card SAS expanders?
Note: HP makes a great PCIe SAS expander. There's an entire community built around its use for storage projects.
The most common case I see is a motherboard or PCIe controller with TWO SFF-8087 ports and a need to connect more than 8 drives split across multiple drive cages. I can't envision something like your scenario where you have 16 ports available on a controller and would want to multiply that via expanders to accommodate more internal drives. If you're working with a server with more than 16 disks, chances are that you'll be encountering a backplane expander somewhere. This would make the PCIe expander unnecessary.
This is all different when you look at external storage, though. The transport there would be a SAS SFF-8088 external cable. Most external drive shelves/JBODs have expanders built-in.
Example RAID configuration output from an HP Smart Array P410i controller connected to two drive cages inside of an HP ProLiant DL370 G6 server through an HP SAS expander card. Note the card is represented as an enclosure (Vendor ID HP, Model HP SAS EXP Card) and has its own SAS WWN.
Smart Array P410i in Slot 0 (Embedded) (sn: 5001438006503640)
array A (Solid State SATA, Unused Space: 0 MB)
logicaldrive 1 (223.5 GB, RAID 0, OK)
physicaldrive 5C:2:8 (port 5C:box 2:bay 8, Solid State SATA, 240.0 GB, OK)
array B (SAS, Unused Space: 0 MB)
logicaldrive 2 (68.3 GB, RAID 1, OK)
physicaldrive 7C:2:1 (port 7C:box 2:bay 1, SAS, 72 GB, OK)
physicaldrive 7C:2:2 (port 7C:box 2:bay 2, SAS, 72 GB, OK)
array C (SATA, Unused Space: 2651025 MB)
logicaldrive 3 (1000.0 GB, RAID 1+0, OK)
logicaldrive 4 (500.0 GB, RAID 1+0, OK)
physicaldrive 2C:1:1 (port 2C:box 1:bay 1, SATA, 1 TB, OK)
physicaldrive 2C:1:2 (port 2C:box 1:bay 2, SATA, 1 TB, OK)
physicaldrive 2C:1:3 (port 2C:box 1:bay 3, SATA, 1 TB, OK)
physicaldrive 2C:1:4 (port 2C:box 1:bay 4, SATA, 1 TB, OK)
physicaldrive 3C:1:5 (port 3C:box 1:bay 5, SATA, 1 TB, OK)
physicaldrive 3C:1:6 (port 3C:box 1:bay 6, SATA, 1 TB, OK)
Expander 250 (WWID: 50014380133153E6, Box: 1)
Enclosure SEP (Vendor ID HP, Model HP SAS EXP Card) 248 (WWID: 50014380133153E5, Box: 2)
SEP (Vendor ID PMCSIERA, Model SRC 8x6G) 249 (WWID: 500143800650364F)
