Understanding Context Managers in Python

So I was working on the evolve command next (hg next). Here my work was with context managers, although I had used context managers before this time I wanted to know a bit more about them.

So I read this blog on Context Managers : https://jeffknupp.com/blog/2016/03/07/python-with-context-managers/

Few takeways:

  • Used to manage resources
  • One of the most famous use : Handling file descriptors

How does a file descriptor leak ?

By not closing the opened files. There is (usually) a limit to the number of file descriptors a process can be assigned. On UNIX-like systems, $ ulimit -n should give you the value of that upper limit. (It’s 1024 on my PC)

  • The general syntax :
with something_that_returns_a_context_manager() as my_resource:
    do_something(my_resource)
    ...
    print('done using my_resource')

Now what returns a context manager ?

  • Lock objects in threading
  • subprocess.Popen
  • pathlib.Path

Basically any object that needs to have close called on it after use is (or should be) a context manager.

How to create a context manager?

The Basic Way

Define a class that contains two special methods: __enter__() and __exit__(). __enter__() returns the resource to be managed (like a file object in the case of open() ). __exit__() does the cleanup work and doesn’t return anything.

Let’s define an Open() context manager using this approach:

class OpenFile():
    def __init__(self,file_name, mode):
        self.file_name = file_name
        self.mode = mode
    def __enter(self):
        self.opened_file = open(self.file_name, self.mode)
        return self.opened_file
    def __exit(self):
        self.opened_file.close()

The contextlib way

Since context managers are so useful there is a Standard Module dedicated to them : contextlib

There is a nice shortcut to define context managers using the @contextmanager decorator. To use @contextmaneger to define context manager:

  • Decorate a generator that yields exactly once.
  • Everything before the call to yield is considered the code for __enter__().
  • Everything after is the code for __exit__().

Let’s rewrite the above OpenFile class using @contextmanager decorator.

@contextmanager
def OpenFile(filename, mode):
    file_ = open(filename, mode)
    yield file_
    file_.close()

Also you can use ContextDecorator to create decorators that in turn will be used to create context managers. Phew!!

comments powered by Disqus