Tuesday, 29 August 2017

The Best Weather Apps for Android

Everyone has at least one weather app installed on their phone, but there are so many different ones out there. Finding the “best” one can be a bit difficult, so we’ve rounded up a couple of the best to help you decide.

How to Buy an Emoji Domain

Emoji are awesome, and even old people (like my 30-year old friends) are starting to realize it now. They add nuance to text messages that you can’t express with boring words. But emoji can do even more than that: now you can register emoji domains. I’ve bought www.🇨🇮.to. Click the link and you get taken to my Twitter page! Let’s look at how.

Logicalis wins top business award for future vision

Top Brazilian business publication chooses Logicalis as “Vision of the Future Company in Brazil for 2017.”London, 29th August, 2017 – Logicalis Group, the international IT solutions and managed services provider, recently announced their Brazilian operation has won recognition for its future vision in a prestigious Latin American business publication.The Época Negócios 360º yearbook is produced in conjunction with world recognised Fundação Dom Cabral, the top business school in Latin America as rated by the Financial... Source: RealWire

Logicalis wins top business award for future vision

Top Brazilian business publication chooses Logicalis as “Vision of the Future Company in Brazil for 2017.”London, 29th August, 2017 – Logicalis Group, the international IT solutions and managed services provider, recently announced their Brazilian operation has won recognition for its future vision in a prestigious Latin American business publication.The Época Negócios 360º yearbook is produced in conjunction with world recognised Fundação Dom Cabral, the top business school in Latin America as rated by the Financial... Source: RealWire

New ADLV Team Eyes Evolving Opportunities Including GDPR

Several new faces have joined the Association for Driving Licence Verification (www.ADLV.co.uk) board. The changes, made in accordance with the Association’s founding charter, see Licence Check’s Terry Hiles elected as Deputy Chair to support the existing Chair, Malcolm Maycock of Licence Bureau. Chris Thornton of DriveTech joins as Treasurer and Mark Sugden of GBG takes over as Secretary and Head of Marketing. Kevin Curtis of Driving Monitor remains as Technical Director. The new team will... Source: RealWire

New ADLV Team Eyes Evolving Opportunities Including GDPR

Several new faces have joined the Association for Driving Licence Verification (www.ADLV.co.uk) board. The changes, made in accordance with the Association’s founding charter, see Licence Check’s Terry Hiles elected as Deputy Chair to support the existing Chair, Malcolm Maycock of Licence Bureau. Chris Thornton of DriveTech joins as Treasurer and Mark Sugden of GBG takes over as Secretary and Head of Marketing. Kevin Curtis of Driving Monitor remains as Technical Director. The new team will... Source: RealWire

New ADLV Team Eyes Evolving Opportunities Including GDPR

Several new faces have joined the Association for Driving Licence Verification (www.ADLV.co.uk) board. The changes, made in accordance with the Association’s founding charter, see Licence Check’s Terry Hiles elected as Deputy Chair to support the existing Chair, Malcolm Maycock of Licence Bureau. Chris Thornton of DriveTech joins as Treasurer and Mark Sugden of GBG takes over as Secretary and Head of Marketing. Kevin Curtis of Driving Monitor remains as Technical Director. The new team will... Source: RealWire

New ADLV Team Eyes Evolving Opportunities Including GDPR

Several new faces have joined the Association for Driving Licence Verification (www.ADLV.co.uk) board. The changes, made in accordance with the Association’s founding charter, see Licence Check’s Terry Hiles elected as Deputy Chair to support the existing Chair, Malcolm Maycock of Licence Bureau. Chris Thornton of DriveTech joins as Treasurer and Mark Sugden of GBG takes over as Secretary and Head of Marketing. Kevin Curtis of Driving Monitor remains as Technical Director. The new team will... Source: RealWire

New ADLV Team Eyes Evolving Opportunities Including GDPR

Several new faces have joined the Association for Driving Licence Verification (www.ADLV.co.uk) board. The changes, made in accordance with the Association’s founding charter, see Licence Check’s Terry Hiles elected as Deputy Chair to support the existing Chair, Malcolm Maycock of Licence Bureau. Chris Thornton of DriveTech joins as Treasurer and Mark Sugden of GBG takes over as Secretary and Head of Marketing. Kevin Curtis of Driving Monitor remains as Technical Director. The new team will... Source: RealWire

New ADLV Team Eyes Evolving Opportunities Including GDPR

Several new faces have joined the Association for Driving Licence Verification (www.ADLV.co.uk) board. The changes, made in accordance with the Association’s founding charter, see Licence Check’s Terry Hiles elected as Deputy Chair to support the existing Chair, Malcolm Maycock of Licence Bureau. Chris Thornton of DriveTech joins as Treasurer and Mark Sugden of GBG takes over as Secretary and Head of Marketing. Kevin Curtis of Driving Monitor remains as Technical Director. The new team will... Source: RealWire

The Best Way to Save Money on Tech: Buy Used

Technology is expensive. We geeks love to shower ourselves with new gadgets, but it’s awfully costly. There are a lot of ways to save money on your tech addiction, but none of them beat buying used.

SimCentric launches Global User Community Forum

On the 28th August 2017 SimCentric Technologies is launching a Global User Community Forum providing a range of supporting tools and resources, an exchange forum to enable file upload and download, and a portal to contact technical support staff directly to ensure that your simulation capability is delivering what you need from it in an agile and responsive manner. To successfully deliver a solution, rather than merely a product, the military and defence industry community... Source: RealWire

oneM2M smart city data chosen to power Hackathon at IEEE event

Data from real-world applications will be used by app developers, product managers and solution architects from across the IoT industryBerlin, Germany, 29 August 2017: Critical data from oneM2M’s successful smart city deployments will be used in the WISE-IoT Consortium’s IoT4Tourism Hackathon. The Hackathon will be held at the IEEE ICCE Berlin Conference, in conjunction with the IFA consumer electronics fair, which will be held September 3-4, 2017.The test bed data – taken from the Busan... Source: RealWire

oneM2M smart city data chosen to power Hackathon at IEEE event

Data from real-world applications will be used by app developers, product managers and solution architects from across the IoT industryBerlin, Germany, 29 August 2017: Critical data from oneM2M’s successful smart city deployments will be used in the WISE-IoT Consortium’s IoT4Tourism Hackathon. The Hackathon will be held at the IEEE ICCE Berlin Conference, in conjunction with the IFA consumer electronics fair, which will be held September 3-4, 2017.The test bed data – taken from the Busan... Source: RealWire

Daniel Bader: Generator Expressions in Python: An Introduction

Generator Expressions in Python: An Introduction

Generator expressions are a high-performance, memory–efficient generalization of list comprehensions and generators. In this tutorial you’ll learn how to use them from the ground up.

Python Generator Expressions Tutorial

In one of my previous tutorials you saw how Python’s generator functions and the yield keyword provide syntactic sugar for writing class-based iterators more easily.

The generator expressions we’ll cover in this tutorial add another layer of syntactic sugar on top—they give you an even more effective shortcut for writing iterators:

With a simple and concise syntax that looks like a list comprehension, you’ll be able to define iterators in a single line of code.

Here’s an example:

iterator = ('Hello' for i in range(3))

Python Generator Expressions 101 – The Basics

When iterated over, the above generator expression yields the same sequence of values as the bounded_repeater generator function we implemented in my generators tutorial. Here it is again to refresh your memory:

def bounded_repeater(value, max_repeats):
    for i in range(max_repeats):
        yield value

iterator = bounded_repeater('Hello', 3)

Isn’t it amazing how a single-line generator expression now does a job that previously required a four-line generator function or a much longer class-based iterator?

But I’m getting ahead of myself. Let’s make sure our iterator defined with a generator expression actually works as expected:

>>> iterator = ('Hello' for i in range(3))
>>> for x in iterator:
...     print(x)
'Hello'
'Hello'
'Hello'

That looks pretty good to me! We seem to get the same results from our one-line generator expression that we got from the bounded_repeater generator function.

There’s one small caveat though:

Once a generator expression has been consumed, it can’t be restarted or reused. So in some cases there is an advantage to using generator functions or class-based iterators.

Generator Expressions vs List Comprehensions

As you can tell, generator expressions are somewhat similar to list comprehensions:

>>> listcomp = ['Hello' for i in range(3)]
>>> genexpr = ('Hello' for i in range(3))

Unlike list comprehensions, however, generator expressions don’t construct list objects. Instead, they generate values “just in time” like a class-based iterator or generator function would.

All you get by assigning a generator expression to a variable is an iterable “generator object”:

>>> listcomp
['Hello', 'Hello', 'Hello']

>>> genexpr
<generator object <genexpr> at 0x1036c3200>

To access the values produced by the generator expression, you need to call next() on it, just like you would with any other iterator:

>>> next(genexpr)
'Hello'
>>> next(genexpr)
'Hello'
>>> next(genexpr)
'Hello'
>>> next(genexpr)
StopIteration

Alternatively, you can also call the list() function on a generator expression to construct a list object holding all generated values:

>>> genexpr = ('Hello' for i in range(3))
>>> list(genexpr)
['Hello', 'Hello', 'Hello']

Of course, this was just a toy example to show how you can “convert” a generator expression (or any other iterator for that matter) into a list. If you need a list object right away, you’d normally just write a list comprehension from the get-go.

Let’s take a closer look at the syntactic structure of this simple generator expression. The pattern you should begin to see looks like this:

genexpr = (expression for item in collection)

The above generator expression “template” corresponds to the following generator function:

def generator():
    for item in collection:
        yield expression

Just like with list comprehensions, this gives you a “cookie-cutter pattern” you can apply to many generator functions in order to transform them into concise generator expressions.

⏰ Sidebar: Pythonic Syntactic Sugar

As I learned more about Python’s iterator protocol and the different ways to implement it in my own code, I realized that “syntactic sugar” was a recurring theme.

You see, class-based iterators and generator functions are two expressions of the same underlying design pattern.

Generator functions give you a shortcut for supporting the iterator protocol in your own code, and they avoid much of the verbosity of class-based iterators. With a little bit of specialized syntax, or syntactic sugar, they save you time and make your life as a developer easier:

This is a recurring theme in Python and in other programming languages. As more developers use a design pattern in their programs, there’s a growing incentive for the language creators to provide abstractions and implementation shortcuts for it.

That’s how programming languages evolve over time—and as developers, we reap the benefits. We get to work with more and more powerful building blocks, which reduces busywork and lets us achieve more in less time.

Filtering Values

There’s one more useful addition we can make to this template, and that’s element filtering with conditions. Here’s an example:

>>> even_squares = (x * x for x in range(10)
                    if x % 2 == 0)

This generator yields the square numbers of all even integers from zero to nine. The filtering condition using the % (modulo) operator will reject any value not divisible by two:

>>> for x in even_squares:
...     print(x)
0
4
16
36
64

Let’s update our generator expression template. After adding element filtering via if-conditions, the template now looks like this:

genexpr = (expression for item in collection
           if condition)

And once again, this pattern corresponds to a relatively straightforward, but longer, generator function. Syntactic sugar at its best:

def generator():
    for item in collection:
        if condition:
            yield expression

In-line Generator Expressions

Because generator expressions are, well…expressions, you can use them in-line with other statements. For example, you can define an iterator and consume it right away with a for-loop:

for x in ('Bom dia' for i in range(3)):
    print(x)

There’s another syntactic trick you can use to make your generator expressions more beautiful. The parentheses surrounding a generator expression can be dropped if the generator expression is used as the single argument to a function:

>>> sum((x * 2 for x in range(10)))
90

# Versus:

>>> sum(x * 2 for x in range(10))
90

This allows you to write concise and performant code. Because generator expressions generate values “just in time” like a class-based iterator or a generator function would, they are very memory efficient.

Too Much of a Good Thing…

Like list comprehensions, generator expressions allow for more complexity than what we’ve covered so far. Through nested for-loops and chained filtering clauses, they can cover a wider range of use cases:

(expr for x in xs if cond1
      for y in ys if cond2
      ...
      for z in zs if condN)

The above pattern translates to the following generator function logic:

for x in xs:
    if cond1:
       for y in ys:
            if cond2:
                ...
                    for z in zs:
                        if condN:
                             yield expr

And this is where I’d like to place a big caveat:

Please don’t write deeply nested generator expressions like that. They can be very difficult to maintain in the long run.

This is one of those “the dose makes the poison” situations where a beautiful and simple tool can be overused to create hard to read and difficult to debug programs.

Just like with list comprehensions, I personally try to stay away from any generator expression that includes more than two levels of nesting.

Generator expressions are a helpful and Pythonic tool in your toolbox, but that doesn’t mean they should be used for every single problem you’re facing. For complex iterators, it’s often better to write a generator function or even a class-based iterator.

If you need to use nested generators and complex filtering conditions, it’s usually better to factor out sub-generators (so you can name them) and then to chain them together again at the top level.

If you’re on the fence, try out different implementations and then select the one that seems the most readable. Trust me, it’ll save you time in the long run.

Generator Expressions in Python – Summary

  • Generator expressions are similar to list comprehensions. However, they don’t construct list objects. Instead, generator expressions generate values “just in time” like a class-based iterator or generator function would.
  • Once a generator expression has been consumed, it can’t be restarted or reused.
  • Generator expressions are best for implementing simple “ad hoc” iterators. For complex iterators, it’s better to write a generator function or a class-based iterator.

What you should know about CephFS

What you should know about CephFS

Today, new storage system interfaces are created regularly to resolve emerging challenges in distributed storage. For example, Amazon Simple Storage Service [S3] (an opaque object store) and Amazon Elastic Block Storage [EBS] (a virtual machine image provider) both provide an essential, scalable storage service within a cloud ecosystem; however even with these new technologies, the conventional file system remains the most-widely used storage interface in computing.


read more

Create versatile visualizations with D3 and Angular

Create versatile visualizations with D3 and Angular

Our world is based on data. We gather it everywhere: forms, feedback, learning techniques, data mining, etc. When it comes to working with that data, we need to do more than show numbers back to our users; we need to make it easy for them to understand what the numbers mean.


read more

How a leader can move forward without consensus

How a leader can move forward without consensus

Open organizations depend on collaboration and inclusion, so when it comes to making decisions, it's natural to wonder how much time and energy we ought to spend in the pursuit of alignment and consensus-building.

Openness and transparency are infused into everything we do at Red Hat, from the way we create technology to our methods of communication. We are a mission-based, purpose-driven organization, and that means company-wide alignment will always be crucial for some of our decisions.


read more

What was your first programming language?

What was your first programming language?

Whether you first learned to program in a classroom setting, on the job, or by teaching yourself, everyone who has contributed code to an open source project has a story of how they first picked up programming. And no matter if you still use it today, your first language played an important role in shaping your understanding of computer systems.

So which language did you begin with?


read more

William Minchin: PhotoSorter Python script 2.1.0 Released

Photosorter is a little Python script to keep my photos from Dropbox organized.

It watches a source directory for modifications and moves new image files to a target directory depending on when the photo was taken, using EXIF data and creation date as a fallback. There is also an option to move existing photos.

Directory and file names follow a simple naming convention (YYYY-MM/YYY_MM_DD/YYYY-MM-DD hh:mm:ss.ext) that keeps everything neatly organized. Duplicates are detected and ignored based on their SHA1 hash and folder path. Photos taken in the same instant get de-duplicated by adding a suffix (-1, -2, etc) to their filenames.

The result looks somewhat like this::

├── 2013-01
│   ├── 2013_01_05
│   │   ├── 2013-01-05\ 13.24.45.jpg
│   │   ├── 2013-01-05\ 14.25.54.jpg
│   │   └── 2013-01-05\ 21.28.48-1.jpg
│   ├── 2013_01_06
│   │   ├── 2013-01-06\ 16.05.02.jpg
│   │   ├── 2013-01-06\ 19.59.25.jpg
│   │   ├── 2013-01-06\ 20.40.28.jpg
│   │   └── 2013-01-06\ 21.14.38.jpg
│   └── 2013_01_08
│       └── 2013-01-08\ 11.45.51.jpg
├── 2013-02
|   └─ ...
├── ...
├── 2013-12
├── 2014-01
├── 2014-02
├── ...
├── 2014-12
├── ...

I use C:\Users\[windows username]\Dropbox\Camera Uploads as the source directory and Z:\Photos as the target. This allows me to move my photo from Dropbox to a local drive, and merge them with the rest of my photo collection.

Impletmentation Notes

I use a different folder set-up that Dan (in his original scirpt) used. The one I’m using matches the default folder set-up for my Canon camera.

Installation

The easiest way to install the script is through pip::

pip install minchin.scripts.photosorter

Requirements

The script’s requirements will be automatically installed in the script is installed via pip as recommended above. They can also be installed manually, if required::

pip install argcomplete>=1.3.0
pip install exifread>=2.1.2
pip install watchdog>=0.8.3

Usage

Watch src_dir and sort incoming photos into dest_dir::

photosorter src_dir dest_dir

When you’re done with it, Ctrl + C will end the program.

If you also want to move the existing files in src_dir (which are, by default, ignored)::

photosorter src_dir dest_dir --move-existing

Known Issues

  • the tests do not currently run.
  • matching (to provide de-duplication) is based on full filepaths matching. I.e. if the per day folder is renamed, the script will not look in the renamed folder for photo matches.
  • Linux deamon setup is untested by myself.

Changes

2.1.0 — 2017-08-28

  • also move MP4 files
  • add changelog to readme

2.0.0 — 2017-08-27

  • move to minchin.scripts.photosorter namespace
  • do releases via minchin.releaser
  • changed generated file folder layout
  • add option to move existing files

License

Distributed under the MIT license. See LICENSE.txt for more information.

Credit

This script is a modified version of the one put together by Dan Bader. Thanks for providing a great template Dan!

How to Create a Local Backup of Your Synology NAS

RELATED: How to Set Up and Get Started with Your Synology NAS

Python Data: Forecasting Time Series data with Prophet – Part 3

This is the third in a series of posts about using Prophet to forecast time series data. The other parts can be found here:

In those previous posts, I looked at forecasting monthly sales data 24 months into the future.   In this post, I wanted to look at using the ‘holiday’ construct found within the Prophet library to try to better forecast around specific events.  If we look at our sales data (you can find it here), there’s an obvious pattern each December.  That pattern could be for a variety of reasons, but lets assume that its due to a promotion that is run every December.   You can see the chart and pattern in the chart below.

sales data plotSales Data – Note the spike every December

Prophet allows you to build a holiday‘ dataframe and use that data in your modeling.  For the purposes of this example, I’ll build my prophet holiday dataframe in the following manner:

promotions = pd.DataFrame({
  'holiday': 'december_promotion',
  'ds': pd.to_datetime(['2009-12-01', '2010-12-01', '2011-12-01', '2012-12-01',
                        '2013-12-01', '2014-12-01','2015-12-01']),
  'lower_window': 0,
  'upper_window': 0,
})

This promotions dataframe consisists of promotion dates for Dec in 2009 through 2015, The lower_window and upper_window values are set to zero to indicate that we don’t want prophet to consider any other months than the ones listed.

Now that I have my promotions dataframe ready to go, I’ll run through the modeling quickly (you can check out the jupyter notebook for more details):

sales_df = pd.read_csv('../examples/retail_sales.csv', index_col='date', parse_dates=True)
df = sales_df.reset_index()
df=df.rename(columns={'date':'ds', 'sales':'y'})
df['y'] = np.log(df['y'])
model = Prophet(holidays=promotions)
model.fit(df);
future = model.make_future_dataframe(periods=24, freq = 'm')
forecast = model.predict(future)
model.plot(forecast);

With these steps, we’ve loaded the data, set it up the way prophet expects and ran our model with the promotions data and then plotted the model, which looks like the following:

Sales Data Modeled with HolidaysSales Data Modeled with Holidays

Given that we have such little data, I doubt the use of holidays will make that much difference in the forecasts, but its a good example to use.  We can check the difference in the model with holidays vs the model without by re-running the prophet forecast without holidays and see that the average difference between the two is ~ 0.06%…which isn’t terribly large, but still worth investigating.  The jupyter notebook that accompanies this post goes into much more detail on this aspect (as well as the overall analysis).

Note: You can find the full code for this post in a Jupyter notebook here:

 

 

The post Forecasting Time Series data with Prophet – Part 3 appeared first on Python Data.

How to Control Your Smarthome Devices with Text Messages

As if there weren’t already enough ways to control all of your smarthome devices, text messaging could be the most convenient method for some users. Here’s how to make it work using IFTTT and some hashtags.

Geek Trivia: Which U.S. State Has The Largest Amount Of Land Set Aside For Native Americans?

Think you know the answer? Click through to see if you're right!

Monday, 28 August 2017

Possbility and Probability: Debugging Flask, requests, curl, and form data

Here’s a recent situation I found myself in where some HTTP form data was not appearing like we expected. Debugging Flask The basic setup is this: A Django process is replaying some HTTP traffic to another system that is written … Continue reading

The post Debugging Flask, requests, curl, and form data appeared first on Possibility and Probability.

Mike Driscoll: Back to School Python Book Sale 2017

It’s time for school and going back to the university, so I am putting on a “Back to School” sale for my Python books. You can now buy my second and third books for 50% off on Leanpub.

You can check out my first book, Python 101, in its entirety over on http://ift.tt/1lHVxvm if you need a sample of my writing style. Leanpub also has samples of both of those books that you can download as a PDF.

Feel free to ask questions in the comments or ping me via the contact form.

How to Stream Music to Your Google Home Over Bluetooth

Google Home is great for taking voice commands, but you can also send music from your phone to its speaker. While most apps let you cast audio to your Google Home, some don’t. For those, you can pair your phone to Google Home via Bluetooth to play music the old-fashioned way. Here’s how to set it up.

How to Remove Third-Party Facebook Apps From Your Account

You know how you can log in to services like Disqus, Best Buy, or Hulu, (or our very own comments) with your Facebook account? It’s a fast and handy way to avoid making a new account and password for every web service under the sun, but that free convenience sometimes comes with a price: access to your data. Companies treasure details of your personal life and the implicit right to send you messages. Here’s how to cut off that connection when you’re done with it.

Chris Moffitt: Building a Bullet Graph in Python

Introduction

Lately I have been spending time reading about various visualization techniques with the goal of learning unique ways to display complex data. One of the interesting chart ideas I have seen is the bullet graph. Naturally, I wanted to see if I could create one in python but I could not find any existing implementations. This article will walk through why a bullet graph (aka bullet chart) is useful and how to build one using python and matplotlib.

Visualization Resources

Over the past few weeks, I have been reading two very good books about data visualization. The first is Cole Nussbaumer Knaflic’s book Storytelling with Data and the second is The Big Book of Dashboards by Steve Wexler, Jeffrey Shaffer and Andy Gotgreave. I found both of these books very enjoyable to read and picked up a lot of useful ideas for developing my own visualizations. This topic is extremely fascinating to me and I think these are nice resources to have in your library.

Storytelling with Data is a guide to presenting data in an effective manner and covers several topics related to choosing effective visuals, telling compelling stories and thinking like a designer. This book does not specifically describe the bullet graph but does introduce some of the concepts and ideas as to why this graph is effective. Because I enjoyed this book so much, I checked out the Storytelling with Data Website which recommends the Big Book of Dashboards book; naturally I had to add it to my library.

The Big Book of Dashboard is an extremely valuable resource for anyone that finds themselves trying to build a dashboard for displaying complex information. In Wexler, Shaffer and Cotgreave’s book, the authors go through an in-depth analysis of 28 different dashboards and explain why they were developed, how they are used and ideas to improve them. The book is very visually appealing and densely packed with great ideas. It is a resource that can be read straight through or quickly browsed through for inspiration.

I have really enjoyed each of these books. I am convinced that there would be a lot better data visualizations in the world if all the Excel and Powerpoint jockeys had both of these books on their desks!

What is a bullet graph?

The Big Book of Dashboards introduced me to the concept of a bullet graph (aka bullet chart) and I found the concept very interesting. I immediately thought of several cases where I could use it.

So, what is a bullet graph? From the book:

“The Bullet Graph encodes data using length/height, position, and color to show actual compared to target and performance bands.”

The example from wikipedia is fairly easy to understand:

Bullet Graph Example

Stephen Few developed the bullet graph to overcome some of the challenges with traditional gauges and meters. The bullet graph is describe by Wikipedia:

The bullet graph features a single, primary measure (for example, current year-to-date revenue), compares that measure to one or more other measures to enrich its meaning (for example, compared to a target), and displays it in the context of qualitative ranges of performance, such as poor, satisfactory, and good. The qualitative ranges are displayed as varying intensities of a single hue to make them discernible by those who are color blind and to restrict the use of colors on the dashboard to a minimum.

Both of these books are tool agnostic so there is not any significant discussion related to how to create these visualizations. I could find examples in Excel but I wanted to see if I could create one in python. I figured if I had existing code that worked, I would be more likely to use it when the time was right. I also like the idea of making this easy to do in python instead of struggling with Excel.

I did some searching but could not find any python examples so I set out to create a reuseable function to build these charts using base matplotlib functionality. I am including the code here and on github in the hope it is useful to others. Feel free to send me pull requests if you have ideas on how to improve it.

Building the chart

The idea for the bullet chart is that we can use a stacked bar chart to represent the various ranges and another smaller bar chart to represent the value. Finally, a vertical line marks the target. Sounds simple enough, right?

Bullet chart meme

Since this is a somewhat complicated layer of components, I think the simplest way to construct this is using matplotlib. In the sections below, I will walk through the basic concepts, then present the final code section which is a little more scalable for multiple charts. I am hoping the community will chime in with better ways to simplify the code or make it more generically useful.

Start the Process

I recommend that you run this code in your jupyter notebook environment. You can access an example notebook here.

To get started, import all the modules we need:

import matplotlib.pyplot as plt
import seaborn as sns
from matplotlib.ticker import FuncFormatter

%matplotlib inline

Astute readers may be wondering why we are including seaborn in the imports. Seaborn has some really useful tools for managing color palettes so I think it is easier to leverage this capability than trying to replicate it in some other manner.

The main reason we need to generate a palette is that we will most likely want to generate a visually appealing color scheme for the various qualitative ranges. Instead of trying to code values by hand, let’s use seaborn to do it.

In this example, we can use the palplot convenience function to display a palette of 5 shades of green:

sns.palplot(sns.light_palette("green", 5))
Green range

Making 8 different shades of purple in reverse order is as easy as:

sns.palplot(sns.light_palette("purple",8, reverse=True))
Purple range

This functionality makes it convenient to create a consistent color scale for as many categories as you need.

Now that we now how to set the palette, let’s try to create a simple bullet graph using the principles laid out in the Effectively Using Matplotlib article.

First, define the values we want to plot:

limits = [80, 100, 150]
data_to_plot = ("Example 1", 105, 120)

This will be used to create 3 ranges: 0-80, 81-100, 101-150 and an “Example” line with a value of 105 and target line of 120. Next, build out a blues color palette:

palette = sns.color_palette("Blues_r", len(limits))

The first step is to build the stacked bar chart of the ranges:

fig, ax = plt.subplots()
ax.set_aspect('equal')
ax.set_yticks([1])
ax.set_yticklabels([data_to_plot[0]])

prev_limit = 0
for idx, lim in enumerate(limits):
    ax.barh([1], lim-prev_limit, left=prev_limit, height=15, color=palette[idx])
    prev_limit = lim

Which yields a nice bar chart:

First example chart

Then we can add a smaller bar chart representing the value of 105:

# Draw the value we're measuring
ax.barh([1], data_to_plot[1], color='black', height=5)

Closer….

Second Example Chart

The final step is to add the target marker using axvline :

ax.axvline(data_to_plot[2], color="gray", ymin=0.10, ymax=0.9)
Third chart

This actually works pretty well but is not very scalable. Ideally we should be able to show multiple bullet graphs on the same scale. Also, this code makes some bad assumptions that do not scale well as the values in the range change.

The Final Code

After much trial and error and playing around with matplotlib, I developed a function that is more generally useful:

def bulletgraph(data=None, limits=None, labels=None, axis_label=None, title=None,
                size=(5, 3), palette=None, formatter=None, target_color="gray",
                bar_color="black", label_color="gray"):
    """ Build out a bullet graph image
        Args:
            data = List of labels, measures and targets
            limits = list of range valules
            labels = list of descriptions of the limit ranges
            axis_label = string describing x axis
            title = string title of plot
            size = tuple for plot size
            palette = a seaborn palette
            formatter = matplotlib formatter object for x axis
            target_color = color string for the target line
            bar_color = color string for the small bar
            label_color = color string for the limit label text
        Returns:
            a matplotlib figure
    """
    # Determine the max value for adjusting the bar height
    # Dividing by 10 seems to work pretty well
    h = limits[-1] / 10

    # Use the green palette as a sensible default
    if palette is None:
        palette = sns.light_palette("green", len(limits), reverse=False)

    # Must be able to handle one or many data sets via multiple subplots
    if len(data) == 1:
        fig, ax = plt.subplots(figsize=size, sharex=True)
    else:
        fig, axarr = plt.subplots(len(data), figsize=size, sharex=True)

    # Add each bullet graph bar to a subplot
    for idx, item in enumerate(data):

        # Get the axis from the array of axes returned when the plot is created
        if len(data) > 1:
            ax = axarr[idx]

        # Formatting to get rid of extra marking clutter
        ax.set_aspect('equal')
        ax.set_yticklabels([item[0]])
        ax.set_yticks([1])
        ax.spines['bottom'].set_visible(False)
        ax.spines['top'].set_visible(False)
        ax.spines['right'].set_visible(False)
        ax.spines['left'].set_visible(False)

        prev_limit = 0
        for idx2, lim in enumerate(limits):
            # Draw the bar
            ax.barh([1], lim - prev_limit, left=prev_limit, height=h,
                    color=palette[idx2])
            prev_limit = lim
        rects = ax.patches
        # The last item in the list is the value we're measuring
        # Draw the value we're measuring
        ax.barh([1], item[1], height=(h / 3), color=bar_color)

        # Need the ymin and max in order to make sure the target marker
        # fits
        ymin, ymax = ax.get_ylim()
        ax.vlines(
            item[2], ymin * .9, ymax * .9, linewidth=1.5, color=target_color)

    # Now make some labels
    if labels is not None:
        for rect, label in zip(rects, labels):
            height = rect.get_height()
            ax.text(
                rect.get_x() + rect.get_width() / 2,
                -height * .4,
                label,
                ha='center',
                va='bottom',
                color=label_color)
    if formatter:
        ax.xaxis.set_major_formatter(formatter)
    if axis_label:
        ax.set_xlabel(axis_label)
    if title:
        fig.suptitle(title, fontsize=14)
    fig.subplots_adjust(hspace=0)

I am not going to go through the code in detail but the basic idea is to create a subplot for each chart and stack them on top of each other. I remove all the spines so that it is relatively clean and simple.

Here is how to use the function to display a “Sales Rep Performance” bullet chart:

data_to_plot2 = [("John Smith", 105, 120),
                 ("Jane Jones", 99, 110),
                 ("Fred Flintstone", 109, 125),
                 ("Barney Rubble", 135, 123),
                 ("Mr T", 45, 105)]

bulletgraph(data_to_plot2, limits=[20, 60, 100, 160],
            labels=["Poor", "OK", "Good", "Excellent"], size=(8,5),
            axis_label="Performance Measure", label_color="black",
            bar_color="#252525", target_color='#f7f7f7',
            title="Sales Rep Performance")
Sales Performance Bullet Chart

I think this is a nice way to compare results across multiple individuals and have a good sense for how they compare to each other. It also shows how values compare to the other quantitative standards we have set. It is illustrative of how much information you can quickly glean from this chart and that trying to do this with other chart types would probably not be as effective.

One other nice thing we can easily do is format the x axis to more consistently display information. In the next case, we can measure marketing budget performance for a hypothetical company. I also chose to keep this in shades of gray and slightly changed the size variable in order to make it look more consistent.

def money(x, pos):
    'The two args are the value and tick position'
    return "${:,.0f}".format(x)

Then create a new set of data to plot:

money_fmt = FuncFormatter(money)
data_to_plot3 = [("HR", 50000, 60000),
                 ("Marketing", 75000, 65000),
                 ("Sales", 125000, 80000),
                 ("R&D", 195000, 115000)]
palette = sns.light_palette("grey", 3, reverse=False)
bulletgraph(data_to_plot3, limits=[50000, 125000, 200000],
            labels=["Below", "On Target", "Above"], size=(10,5),
            axis_label="Annual Budget", label_color="black",
            bar_color="#252525", target_color='#f7f7f7', palette=palette,
            title="Marketing Channel Budget Performance",
            formatter=money_fmt)
Budget Performance Bullet Graph

Summary

The proliferation of data and data analysis tools has made the topic of visualization very important and is a critical skill for anyone that does any level of data analysis. The old world of Excel pie charts and 3D graphs is not going to cut it going forward. Fortunately there are many resources to help build that skill. The Big Book of Dashboards and Storytelling with Data are two useful resources that are worth adding to your library if you do any level of data visualization.

The Big Book of Dashboards introduced me to the bullet graph which is a useful format for displaying actual results vs various targets and ranges. Unfortunately there was not an existing python implementation I coudl find. The fairly compact function described in this article is a good starting point and should be a helpful function to use when creating your own bullet graphs.

Feel free to send github pull requests if you have ideas to make this code more useful.

Doug Hellmann: smtplib — Simple Mail Transfer Protocol Client — PyMOTW 3

smtplib includes the class SMTP , which can be used to communicate with mail servers to send mail. Read more… This post is part of the Python Module of the Week series for Python 3. See PyMOTW.com for more articles from the series.