Data Tools - Dealing With The FRED HTTPS Migration In Python

The FRED Economic Data System provided by the St. Louis Fed has a programming interface (an API)  which I use to stock the database in my personal research system. The interface is switching over to use the secure HTTPS protocol (from HTTP), and this means that code needs to be updated. I use Python to access the database, and I ran into issues. I am now describing how I worked around the problems.

I assume that the reader knows Python; this is not a tutorial on how to use the fred package in Python. (I expect that I will eventually write tutorials on those lines.)

Although I have done a fair amount of Python programming, I have not kept up with the changes in how packages are distributed. I do not know whether there was a problem with just my installation, or there is a more general problem with the "fred" Python package on Python 3.3 (which is no longer the latest version).

The upgrade process gave me some unusual warnings, but it appeared that it worked.

However, when I tried downloading data, I ran into issues. For example, if I tried "fred.category()", I got an error of the following format:

TypeError: can't use a string pattern on a bytes-like object

What is happening is that the data is not being decoded properly within the fred package. I needed to switch over to using XML encoding. For example, I used "fred.category(xml=True)", it downloads the data properly. The "xml=True" option should be added to any other fred calls.

The problem was that my original code used the default format, which returned the data as a Python dict (dictionary) object. I fixed this by passing the fred output through the parse() command in the xmltodict package.

>>> out = fred.category(xml=True)
>>> import xmltodict
>>> out_dict = xmltodict.parse(out)

The only difference from the original version was that fields in the dictionary have "@" appended in front of their labels. There may have been a way of cleaning that up, but it probably would have required reading some documentation.

(c) Brian Romanchuk 2015