execute *.sql file with python MySQLdb

This worked for me:

with open('schema.sql') as f:
    cursor.execute(f.read().decode('utf-8'), multi=True)

I also needed to execute a SQL file, but the catch was that there wasn't one statement per line, so the accepted answer didn't work for me.

The SQL file I wanted to execute looked like this:

-- SQL script to bootstrap the DB:
--
CREATE USER 'x'@'%' IDENTIFIED BY 'x';
GRANT ALL PRIVILEGES ON mystore.* TO 'x'@'%';
GRANT ALL ON `%`.* TO 'x'@`%`;
FLUSH PRIVILEGES;
--
--
CREATE DATABASE oozie;
GRANT ALL PRIVILEGES ON oozie.* TO 'oozie'@'localhost' IDENTIFIED BY 'oozie';
GRANT ALL PRIVILEGES ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie';
FLUSH PRIVILEGES;
--
USE oozie;
--
CREATE TABLE `BUNDLE_ACTIONS` (
  `bundle_action_id` varchar(255) NOT NULL,
  `bundle_id` varchar(255) DEFAULT NULL,
  `coord_id` varchar(255) DEFAULT NULL,
  `coord_name` varchar(255) DEFAULT NULL,
  `critical` int(11) DEFAULT NULL,
  `last_modified_time` datetime DEFAULT NULL,
  `pending` int(11) DEFAULT NULL,
  `status` varchar(255) DEFAULT NULL,
  `bean_type` varchar(31) DEFAULT NULL,
  PRIMARY KEY (`bundle_action_id`),
  KEY `I_BNDLTNS_DTYPE` (`bean_type`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
--
--

Some statements in the above file lie on a single line and some statements also span multiple lines (like the CREATE TABLE at the end). There are also a few SQL inline comment lines that begin with "--".

As suggested by ThomasK, I had to write some simple rules to join lines into a statement. I ended up with a function to execute a sql file:

def exec_sql_file(cursor, sql_file):
    print "\n[INFO] Executing SQL script file: '%s'" % (sql_file)
    statement = ""

    for line in open(sql_file):
        if re.match(r'--', line):  # ignore sql comment lines
            continue
        if not re.search(r';$', line):  # keep appending lines that don't end in ';'
            statement = statement + line
        else:  # when you get a line ending in ';' then exec statement and reset for next statement
            statement = statement + line
            #print "\n\n[DEBUG] Executing SQL statement:\n%s" % (statement)
            try:
                cursor.execute(statement)
            except (OperationalError, ProgrammingError) as e:
                print "\n[WARN] MySQLError during execute statement \n\tArgs: '%s'" % (str(e.args))

            statement = ""

I'm sure there's scope for improvement, but for now it's working pretty well for me. Hope someone finds it useful.


From python, I start a mysql process to execute the file for me:

from subprocess import Popen, PIPE
process = Popen(['mysql', db, '-u', user, '-p', passwd],
                stdout=PIPE, stdin=PIPE)
output = process.communicate('source ' + filename)[0]

Tags:

Python

Mysql