Select array element from Spark Dataframes split method in same call?

Use getItem. I'd say don't use python UDF just to make the code looks prettier - it's much slower than native DataFrame functions (due to moving data between python and JVM).

from pyspark.sql import functions as F
df.select(F.split(df.http_request, '/').alias('http').getItem(1))