How do I extend a python module? Adding new functionality to the `python-twitter` package

A few ways.

The easy way:

Don't extend the module, extend the classes.

exttwitter.py

import twitter

class Api(twitter.Api):
    pass 
    # override/add any functions here.

Downside : Every class in twitter must be in exttwitter.py, even if it's just a stub (as above)

A harder (possibly un-pythonic) way:

Import * from python-twitter into a module that you then extend.

For instance :

basemodule.py

 class Ball():
    def __init__(self,a):
        self.a=a
    def __repr__(self):
        return "Ball(%s)" % self.a

def makeBall(a):
    return Ball(a)

def override():
    print "OVERRIDE ONE"

def dontoverride():
    print "THIS WILL BE PRESERVED"

extmodule.py

from basemodule import *
import basemodule

def makeBalls(a,b):
    foo = makeBall(a)
    bar = makeBall(b)
    print foo,bar

def override():
    print "OVERRIDE TWO"

def dontoverride():
    basemodule.dontoverride()
    print "THIS WAS PRESERVED"

runscript.py

import extmodule

#code is in extended module
print extmodule.makeBalls(1,2)
#returns Ball(1) Ball(2)

#code is in base module
print extmodule.makeBall(1)
#returns Ball(1)

#function from extended module overwrites base module
extmodule.override()
#returns OVERRIDE TWO

#function from extended module calls base module first
extmodule.dontoverride()
#returns THIS WILL BE PRESERVED\nTHIS WAS PRESERVED

I'm not sure if the double import in extmodule.py is pythonic - you could remove it, but then you don't handle the usecase of wanting to extend a function that was in the namespace of basemodule.

As far as extended classes, just create a new API(basemodule.API) class to extend the Twitter API module.

Don't add them to the module. Subclass the classes you want to extend and use your subclasses in your own module, not changing the original stuff at all.

Here’s how you can directly manipulate the module list at runtime – spoiler alert: you get the module type from types module:

from __future__ import print_function
import sys
import types
import typing as tx

def modulize(namespace: tx.Dict[str, tx.Any],
             modulename: str,
             moduledocs: tx.Optional[str] = None) -> types.ModuleType:

    """ Convert a dictionary mapping into a legit Python module """

    # Create a new module with a trivially namespaced name:
    namespacedname: str = f'__dynamic_modules__.{modulename}'
    module = types.ModuleType(namespacedname, moduledocs)
    module.__dict__.update(namespace)

    # Inspect the new module:
    name: str = module.__name__
    doc: tx.Optional[str] = module.__doc__
    contents: str = ", ".join(sorted(module.__dict__.keys()))
    print(f"Module name:      {name}")
    print(f"Module contents:  {contents}")
    if doc:
        print(f"Module docstring: {doc}")

    # Add to sys.modules, as per import machinery:
    sys.modules.update({ modulename : module })

    # Return the new module instance:
    return module

… you could then use such a function like so:

ns = {
         'func' : lambda: print("Yo Dogg"), # these can also be normal non-lambda funcs
    'otherfunc' : lambda string=None: print(string or 'no dogg.'),
      '__all__' : ('func', 'otherfunc'),
      '__dir__' : lambda: ['func', 'otherfunc'] # usually this’d reference __all__
}

modulize(ns, 'wat', "WHAT THE HELL PEOPLE")
import wat

# Call module functions:
wat.func()
wat.otherfunc("Oh, Dogg!")

# Inspect module:
contents = ", ".join(sorted(wat.__dict__.keys()))
print(f"Imported module name:      {wat.__name__}")
print(f"Imported module contents:  {contents}")
print(f"Imported module docstring: {wat.__doc__}")

… You could also create your own module subclass, by specifying types.ModuleType as the ancestor of your newly declared class, of course; I have never personally found this necessary to do.

(Also, you don’t have to get the module type from the types module – you can always just do something like ModuleType = type(os) after importing os – I specifically pointed out this one source of the type because it is non-obvious; unlike many of its other builtin types, Python doesn’t offer up access to the module type in the global namespace.)

The real action is in the sys.modules dict, where (if you are appropriately intrepid) you can replace existing modules as well as adding your new ones.