Setting Up Anaconda Python WSGI Apps On IIS

After unconsciously avoiding IIS 8.5 during nearly six months at Microsoft, I had to deploy a few simple Python apps on Windows Server. Nothing against it, really, but I’m much more focused on microservices these days, and as such the only Windows web server I play with is Kestrel, given that it’s the future for ASP.NET.

But since I had to use IIS, I went about it in my usual pragmatic fashion – i.e., made absolutely sure it would be as painless as possible in the future. In the process, I also partially duplicated the setup used by Azure App Service Web Apps, since I wanted to be able to deploy on both with minimal changes.

Most of the work I do on Azure these days involves Big Data or data science of some sort (usually with a machine learning twist), and as such I tend to use the Data Science Virtual Machine image.

That image has a bunch of essential goodies installed, beginning with Continuum Analytics’ Anaconda Python distribution, the latest R Server, and other staple Microsoft tools like Visual Studio and SQL Server Express – in short, all you need to start munging data at your leisure, including Jupyter and Power BI for trying out scripts interactively and making sense of data during the whole project.

I’ve taken to installing the R kernel into Jupyter, so I can do just about everything using notebooks and share the results easily – but that’s a topic for another post.

I wanted to expose a number of REST endpoints to publish data from a model I’m developing, and wanted to test them locally using the Anaconda runtime before bundling up the lot and pushing them out to an Azure Web App.

I’d rather use Anaconda because it’s already installed, and also because I’ve had mixed results with the Python for Windows binaries from python.org over the years – nothing much, really, but installing another interpreter seemed like an unnecessary hassle.

Azure App Service leverages the web.config file and a custom WSGI handler to make it easy to set up Python apps with minimal fuss (including activating a virtualenv to allow for custom packages), but I wanted to have as close a setup as possible using Anaconda’s Python interpreter and packages.

Here’s how things work in this setup, which emulates what Azure does for Python apps:

• web.config sets up IIS to use wfastcgi as a FastCGI request handler (with a rewrite rule to make everything go through FastCGI) and defines WSGI_HANDLER, PYTHONPATH and a number of other environment variables, including WSGI_ALT_VIRTUALENV_HANDLER
• wfastcgi then loads and looks up WSGI_HANDLER, which points to ptvs_virtualenv_proxy.py
• That proxy sets up WSGI logging, boots your virtualenv and then loads up WSGI_ALT_VIRTUALENV_HANDLER, which is actually your app

In this case, there’s no need for the runtime.txt file that Azure uses to determine the runtime, but it’s worth pointing out it exists.

To make this work locally, all you need to do is install wfastcgi inside Anaconda and pick your WSGI framework (I went with Bottle, which is what I use for most stuff, but using Django instead is child’s play).

There’s a hidden kink, though, which is that I had to install the rewrite module manually. That had me stumped for a while, since I stopped using IIS since 6.0(ish).

To set everything up from scratch on a fresh machine:

• Set up IIS as usual, enabling CGI (to get the FastCGI adapter)
• Install the rewrite module
• Enable web.config overrides (I edited %windir%\System32\inetsrv\config\applicationHost.config “the hard way” using Notepad with admin privileges, but you can use the appcmd.exe CLI tool)
• Drop my web.config, myapp.py and a tweaked version of pvts_virtualenv_proxy.py into the default website (I actually nuked the default while experimenting, but I’m already running two different sites on the same box with this setup)
• Change permissions on the log folder to enable SERVERNAME\IIS_IUSRS to write to it
• Run pip install bottle wfastcgi
• Run wfastcgi-enable as an administrator

…and bingo, the little Bottle app should just work.

Here’s the web.config file. Note the logging and WSGI handler paths:

<?xml version="1.0"?>
<configuration>
  <appSettings>
    <add key="WSGI_HANDLER" value="ptvs_virtualenv_proxy.handler"/>
    <!-- Make sure HOSTNAME\IIS_IUSRS can write to this -->
    <add key="WSGI_LOG" value="c:\inetpub\logs\logfiles\w3svc1\wsgi.txt"/>
    <add key="PYTHONPATH" value="c:\inetpub\pyroot\default" />
    <add key="WSGI_ALT_VIRTUALENV_HANDLER" value="myapp.app" />
    <add key="WSGI_ALT_VIRTUALENV_ACTIVATE_THIS" value="c:\inetpub\pyroot\default\env\Scripts\activate_this.py" />
  </appSettings>
  <system.web>
    <compilation debug="true" targetFramework="4.0" />
  </system.web>
  <system.webServer>
    <modules runAllManagedModulesForAllRequests="true" />
    <handlers>
      <add name="Python FastCGI"
           path="handler.fcgi"
           verb="*"
           modules="FastCgiModule"
           scriptProcessor="c:\Anaconda\python.exe|c:\Anaconda\Lib\site-packages\wfastcgi.pyc"
           resourceType="Unspecified"
           requireAccess="Script" />
    </handlers>
    <!-- 
        this requires the rewrite module, available at http://www.iis.net/learn/extensions/url-rewrite-module/using-the-url-rewrite-module
        and tweaking C:\Windows\System32\inetsrv\config\applicationHost.config 
        
        %windir%\system32\inetsrv\appcmd.exe unlock config -section:system.webServer/handlers
        %windir%\system32\inetsrv\appcmd.exe unlock config -section:system.webServer/modules
     -->
    <rewrite>
      <rules>
        <rule name="Configure Python" stopProcessing="true">
          <match url="(.*)" ignoreCase="false" />
          <action type="Rewrite" url="handler.fcgi/{R:1}" appendQueryString="true" />
        </rule>
      </rules>
    </rewrite>
  </system.webServer>
</configuration>

Here’s the slightly tweaked ptvs_virtualenv_proxy.py. The virtualenv bit is commented out for this minimal setup, but it’s a simple matter to re-enable.

import os
import datetime

def log(txt):
    """Logs fatal errors to a log file if WSGI_LOG env var is defined"""
    log_file = os.environ.get('WSGI_LOG')
    if log_file:
        f = file(log_file, 'a+')
        try:
            f.write(str(datetime.datetime.now()))
            f.write(': ')
            f.write(txt)
        finally:
          f.close()

def get_wsgi_handler(handler_name):
      if not handler_name:
          raise Exception('WSGI_ALT_VIRTUALENV_HANDLER env var must be set')
    
      module, _, callable = handler_name.rpartition('.')
      if not module:
          raise Exception('WSGI_ALT_VIRTUALENV_HANDLER must be set to module_name.wsgi_handler, got %s' % handler_name)
    
      if isinstance(callable, unicode):
          callable = callable.encode('ascii')

      if callable.endswith('()'):
          callable = callable.rstrip('()')
          handler = getattr(__import__(module, fromlist=[callable]), callable)()
      else:
          handler = getattr(__import__(module, fromlist=[callable]), callable)
    
      if handler is None:
          raise Exception('WSGI_ALT_VIRTUALENV_HANDLER "' + handler_name + '" was set to None')
            
      return handler
      
# Uncomment when virtualenv is required
#activate_this = os.getenv('WSGI_ALT_VIRTUALENV_ACTIVATE_THIS')
#if activate_this is None:
#    raise Exception('WSGI_ALT_VIRTUALENV_ACTIVATE_THIS is not set')
#log('doing activation' + '\n')
#execfile(activate_this, dict(__file__=activate_this))

log('getting handler ' + os.getenv('WSGI_ALT_VIRTUALENV_HANDLER') + '\n')
handler = get_wsgi_handler(os.getenv('WSGI_ALT_VIRTUALENV_HANDLER'))
log('got handler ' + repr(handler))

…and, finally, the Bottle app:

import bottle
from bottle import route

@route('/')
def index():                                                                    
     return "Hello World!"

app = bottle.app()

To deploy the same thing on Azure, all you’ll need to do is strip out the modified web.config and virtualenv proxy, set up a git remote to your Azure Web App, and push away.

I’ll make sure to update this when I build something complex enough to warrant a virtualenv – If you’ve been keeping track, I am rather picky about how I manage dependencies, regardless of platform, but I’m quite partial to the new wheel format, and that’s probably the most sensible way to deploy packages with native bindings on Windows.