Working with a database

For me, the best way to learn how to do it was in Wolfram DatabaseLink User Guide. You can download the PDF for free.

Here is some Mathematica code example for insert and select from Microsoft SQL Server.

(*Function for Connection String*)
openConn[]:= 
  OpenSQLConnection[
   JDBC["Microsoft SQL Server(jTDS)", "myIpNumber"], 
   "Username" -> "myUsername", 
   "Password" -> "myPassword" ]

(*Insert Data*)
insertData[newData_]:=Module[{conn=openConn[],columnsName},
    columnsName={"col1","col2","col3"};
    SQLInsert[conn,"myBaseName",columnsName,newData];
    CloseSQLConnection[conn];
]

(*Select Data*)
topNData[n_]:=Module[{conn=openConn[], data, columnsName, sql,r},
    sql="Select top "<>ToString@n<>" * from myBaseName";
    r=SQLExecute[conn,sql];
    CloseSQLConnection[conn];
    r
]

I would rule out option #1 as it would be like working inside a spreadsheet but using only cell A1.

For inserting large amounts of data I recommend you to just use DatabaseLink's SQLExecute.

As your dataset is large, and you want to insert this as fast as possible please take into account that there are very large differences in performance depending on how you do this.

Lets see some examples. Start by creating some sample data:

data = RandomInteger[100, {20000, 1}];

So data contains 20000 random integers. Lets assume that your database connection is in the variable conn and that there is a table myTable in your database with a single column which can hold integer numbers.

Here we see that a very fast operation repeated 20k times can take a long time:

Scan[SQLExecute[conn, "INSERT INTO myTable VALUES (`1`) ", {#}] &, 
  Flatten[data]] // Timing

{323.609754, Null}

Instead you can send all of the numbers at the same time to DatabaseLink for insertion which results in a much better performance:

SQLExecute[conn, "INSERT INTO myTable VALUES (`1`) ", data]; // Timing

{3.961870, Null}

We can also try to send the numbers in batches of 100 numbers

Scan[SQLExecute[conn, "INSERT INTO myTable VALUES (`1`) ", #] &, 
  Partition[data, 100, 100, 1, {}]] // Timing

{7.210615, Null}

And finally we can build a huge query string:

query = "INSERT INTO myTable VALUES " <> 
    StringReplace[
     StringTake[ToString@data, {2, -2}], {"{" -> "(", 
      "}" -> ")"}]; // Timing

{0.039831, Null}

And execute it at a blazing fast speed:

SQLExecute[conn, query]; // Timing

{0.025865, Null}

This last approach must have a downside. I guess it uses more RAM.