Data Science, Machine Learning, Natural Language Processing, Text Analysis, Recommendation Engine, R, Python
Thursday, 31 August 2017
Meet the millennials who mine - Not gold but Cryptocurrency
How to Regain Access to Your Plex Server When You’re Locked Out
Python Software Foundation
Python in Nigeria
| Aisha Bello |
| Chukwudi Nwachukwu |
| Smart Girls Project |
| Django Girls Lagos |
Funding for Events
| Django Girls Lagos |
Anker’s Eufy Genie vs. Amazon Echo Dot: Are the Savings Worth It?
Geek Trivia: The Medical Term For The Numbness You Experience When A Limb “Falls Asleep” Is?
Wednesday, 30 August 2017
Dataquest: Python Cheat Sheet for Data Science: Intermediate
The tough thing about learning data is remembering all the syntax. While at Dataquest we advocate getting used to consulting the Python documentation, sometimes it’s nice to have a handy reference, so we’ve put together this cheat sheet to help you out!
This cheat sheet is the companion to our Python Basics Data Science Cheat Sheet
If you’re interested in learning Python, we have a free Python Programming: Beginner course which can start you on your data science journey.
Download a Printable PDF of this Cheat Sheet
Key Basics, Printing and Getting Help
This cheat sheet assumes you are familiar with the content of our Python Basics Cheat Sheet
s |
A Python string variable |
i |
A Python integer variable |
f |
A Python float variable |
l |
A Python list variable |
d |
A Python dictionary variable |
Lists
l.pop(3) |
Returns the fourth item from l and deletes... |
How to Search Your Amazon Order History
Riyadh-headquartered Saudi Rotorcraft Support Company (SRSC) partners with Ramco Systems to provide national MRO Software Support across Saudi Arabia
The Cheapest Ways To Stream College Football (Without Cable)
Continuum Analytics Blog: Anaconda: A Coming of Age Story
Read more →
Matthew Rocklin: Dask Release 0.15.2
This work is supported by Anaconda Inc. and the Data Driven Discovery Initiative from the Moore Foundation.
I’m pleased to announce the release of Dask version 0.15.2. This release contains stability enhancements and bug fixes. This blogpost outlines notable changes since the 0.15.0 release on June 11th.
You can conda install Dask:
conda install dask
or pip install from PyPI
pip install dask[complete] --upgrade
Conda packages are available both on the defaults and conda-forge channels.
Full changelogs are available here:
Some notable changes follow.
New dask-core and dask conda packages
On conda there are now three relevant Dask packages:
dask-core: Package that includes only the core Dask package. This has no dependencies other than the standard library. This is primarily intended for down-stream libraries that depend on certain parts of Dask.distributed: Dask’s distributed scheduler, depends on Tornado, cloudpickle, and other libraries.dask: Metapackage that includes dask-core, distributed, and all relevant libraries like NumPy, Pandas, Bokeh, etc.. This is intended for users to install
This organization is designed to both allow downstream libraries to only depend on the parts of Dask that they need while also making the default behavior for users all-inclusive.
Downstream libraries may want to change conda dependencies from dask to dask-core. They will then need to be careful to include the necessary libraries (like numpy or cloudpickle) based on their user community.
Improved Deployment
Due to increased deployment on Docker or other systems with complex networking rules dask-worker processes now include separate --contact-address and --listen-address keywords that can be used to specify addresses that they advertise and addresses on which they listen. This is especially helpful when the perspective of ones network can shift dramatically.
dask-worker scheduler-address:8786 \
--contact-address 192.168.0.100:9000 # contact me at 192.168.0.100:9000
--listen-address 172.142.0.100:9000 # I listen on this host
Additionally other services like the HTTP and Bokeh servers now respect the hosts provided by --listen-address or --host keywords and will not be visible outside of the specified network.
Avoid memory, file descriptor, and process leaks
There were a few occasions where Dask would leak resources in complex situations. Many of these have now been cleaned up. We’re grateful to all those who were able to provide very detailed case studies that demonstrated these issues and even more grateful to those who participated in resolving them.
There is undoubtedly more work to do here and we look forward to future collaboration.
Array and DataFrame APIs
As usual, Dask array and dataframe have a new set of functions that fill out their API relative to NumPy and Pandas.
See the full APIs for further reference:
Deprecations
Officially deprecated dask.distributed.Executor, users should use dask.distributed.Client instead. Previously this was set to an alias.
Removed Bag.concat, users should use Bag.flatten instead.
Removed magic tuple unpacking in Bag.map like bag.map(lambda x, y: x + y). Users should unpack manually instead.
Julia
Developers from the Invenia have been building Julia workers and clients that operate with the Dask.distributed scheduler. They have been helpful in raising issues necessary to ensure cross-language support.
- Link: http://ift.tt/2wnW1mD
- Relevant issue: dask/distribugted #586
Acknowledgements
The following people contributed to the dask/dask repository since the 0.15.0 release on June 11th
- Bogdan
- Elliott Sales de Andrade
- Bruce Merry
- Erik Welch
- Fabian Keller
- James Bourbeau
- Jeff Reback
- Jim Crist
- John A Kirkham
- Luke Canavan
- Mark Dunne
- Martin Durant
- Matthew Rocklin
- Olivier Grisel
- Søren Fuglede Jørgensen
- Stephan Hoyer
- Tom Augspurger
- Yu Feng
The following people contributed to the dask/distributed repository since the 1.17.1 release on June 14th:
- Antoine Pitrou
- Dan Brown
- Elliott Sales de Andrade
- Eric Davies
- Erik Welch
- Evan Welch
- John A Kirkham
- Jim Crist
- James Bourbeau
- Jeremiah Lowin
- Julius Neuffer
- Martin Durant
- Matthew Rocklin
- Paul Anton Letnes
- Peter Waller
- Sohaib Iftikhar
- Tom Augspurger
Additionally we’re happy to announce that John Kirkham (@jakirkham) has accepted commit rights to the Dask organization and become a core contributor. John has been active through the Dask project, and particularly active in Dask.array.
How to Enable or Disable Notification Dots in Android 8.0 Oreo
Amulet Hotkey Delivers Unique Virtual GPU for Dell EMC PowerEdge servers
3 open source Python GUI frameworks
There comes a time in the journey of most any programmer when they are ready to branch out past the basic examples and start to build a graphical interface to their program.
In Python, the steps to get started with GUI programming are not terribly complex, but they do require the user to begin making some choices. By its nature as a general purpose programming language with interpreters available across every common operating system, Python has to be fairly agnostic as to the choices it presents for creating graphical user interfaces.
read more
What’s the Difference Between a Facebook Profile, Page, and Group?
App Subscriptions Are a Good Thing
Industry 4.0: How to manage the change and prepare leaders
Opportunities for blockchain in healthcare and pharma
8 best practices for robotic process automation
GDPR: What Does it Mean for Businesses?
8 open source Android apps for education
With a new school year beginning in many areas, now is a great time for parents, students, and educators to try out apps that can help with the learning experience. The following eight apps can turn an Android phone or tablet into a tool to help a student learn and study. They are all open source and all of them are available from the F-Droid repository, so you easily add these apps to any phone that is configured to allow apps from external repositories.
read more
How to create reproducible build environments with Rebuild
Building modern software in a predictable and repeatable way isn't easy. The overwhelming number of software dependencies and the need to isolate conflicting components presents numerous challenges in managing build environments.
Although there are many tools aimed at mitigating this challenge, there are two approaches most of them take: either they rely on package managers to preserve and replicate package sets, or they use virtual or physical machines with preconfigured environments.
read more
Mentoring: Your path to immortality
Have you ever wondered why some open source projects have better luck than others when it comes to attracting and retaining enthusiastic contributors? Here are a few ways open source projects can improve their chances of getting—and keeping—the kinds of contributors who help make projects succeed.
read more
Google, Apple face off over augmented reality technology
RTI activist says Aadhaar contract gave foreign firms access to un-encrypted data
Can electric scooter maker Okinawa sell a Japanese dream to petrol-obsessed Indians?
States act on Aadhaar, link it to local schemes
Weekly Python Chat: Virtual Environments in Python
What's a virtual environment? Do you really need them? Is there any way to make them easier to use?
If you're using Python, you probably should be using virtual environments. In this chat we'll discuss what virtual environments are and how to use them.
How Do You Define a PowerShell Function That Requires Elevation?
How to Set Up the Kwikset Kevo Fob to Unlock Your Door Without Your Phone
Geek Trivia: Residential Communities Where People Live With Their Planes And Direct Runway Access Are Called?
Tuesday, 29 August 2017
Tomasz Früboes: Multiprocessing – understand your exception
DataCamp: Python Seaborn Cheat Sheet For Statistical Data Visualization
You most probably will know by now that data storytelling, accomplished by data visualization, amongst other things, is an essential skill for every data scientist: after you have turned the raw data into understanding, insights and knowledge, you also need to communicate these findings effectively to your audience.
For most beginners, the first Python data visualization library that they use is, naturally, Matplotlib. It is a Python 2D plotting library that enables users to make publication-quality figures. It is quite an extensive library where a cheat sheet will definitely come in handy when you're learning, but when you manage to use this library effectively, you'll also be able to get insights and work better with other packages, such as Pandas, that intend to build more plotting integration with Matplotlib as time goes on.
Another package that you'll be able to tackle is Seaborn, the statistical data visualization library of Python.
DataCamp has created a Seaborn cheat sheet for those who are ready to get started with this data visualization library with the help of a handy one-page reference.
You'll see that this cheat sheet presents you with the five basic steps that you can go through to make beautiful statistical graphs in Python.
Check out the infographic by clicking on the button below:
This cheat sheet will walk you through the five steps that you need to go through to make these plots: you'll see how you can load in data, set the figure aesthetics, plot, customize and eventually, show or save your plot with Seaborn.
What might have looked difficult before will definitely be more clear once you start using this cheat sheet! Use it in combination with the Seaborn Gallery, the documentation and our tutorial.
Also, don't miss out on our other cheat sheets for data science that cover SciPy, Numpy, Scikit-Learn, Bokeh, Pandas and the Python basics.
How to Stop Netflix From Sending You Emails and Notifications
Finn Partners Selected To Publicise Not-For-Profit Cloud Platform, The Good Exchange
Take the 2017 Jenkins Survey!
| This is a guest post by Brian Dawson on behalf of CloudBees, where he works as a DevOps Evangelist responsible for developing and sharing continuous delivery and DevOps best practices. He also serves as the CloudBees Product Marketing Manager for Jenkins. |
Once again it’s that time of year when CloudBees sponsors the Jenkins Community Survey to assist the community with gathering objective insights into how jenkins is being used and what users would like to see in the Jenkins project.
Your personal information (name, email address and company) will NOT be used by CloudBees for sales or marketing.
As an added incentive to take the survey, CloudBees will enter participants into a drawing for a free pass to Jenkins World 2018 (1st prize) and a $100 Amazon Gift Card (2nd prize). The survey will close at the end of September, so click the link at the end of the blog post to get started!
All participants will be able to access reports summarizing survey results. If you’re curious about what insights your input will provide, see the results of last year’s 2016 survey:
Your feedback helps capture a bigger picture of community trends and needs. There are laws that govern prize giveaways and eligibility; CloudBees has compiled all those fancy terms and conditions here.
Please take the survey and let your voice be heard - it will take less than 10 minutes.
How to Train Your Pandora Radio Stations Better
Real Python: User Authentication with Angular 4 and Flask
In this tutorial, we’ll demonstrate how to set up token-based authentication (via JSON Web Tokens) with Angular 4 and Flask.
Main Dependencies:
Auth Workflow
Here’s the full user auth process:
- Client logs in and the credentials are sent to the server
- If the credentials are correct, the server generates a token and sends it as a response to the client
- Client receives and stores the token in Local Storage
- Client then sends token to server on subsequent requests within the request header
Project Setup
Start by globally installing the Angular CLI:
1
|
|
Then generate a new Angular 4 project boilerplate:
1
|
|
Fire up the app after the dependencies install:
1 2 |
|
It will probably take a minute or two to compile and build your application. Once done, navigate to http://localhost:4200 to ensure the app is up and running.
Open up the project in your favorite code editor, and then glance over the code:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
|
In short, the client-side code lives in the “src” folder and the Angular app itself can be found in the “app” folder.
Take note of the AppModule within app.module.ts. This is used to bootstrap the Angular app. The @NgModule decorator takes metadata that lets Angular know how to run the app. Everything that we create in this tutorial will be added to this object.
Make sure you have a decent grasp of the app structure before moving on.
NOTE: Just getting started with Angular 4? Review the Angular Style Guide, since the app generated from the CLI follows the recommended structure from that guide, as well as the Angular4Crud Tutorial.
Did you notice that the CLI initialized a new Git repo? This part is optional, but it’s a good idea to create a new Github repository and update the remote:
1
|
|
Now, let’s wire up a new component…
Auth Component
First, use the CLI to generate a new Login component:
1
|
|
This set up the component files and folders and even wired it up to app.module.ts. Next, let’s change the login.component.ts file to the following:
1 2 3 4 5 6 7 8 9 10 11 |
|
If you haven’t used TypeScript before, then this code probably looks pretty foreign to you. TypeScript is a statically-typed superset of JavaScript that compiles to vanilla JavaScript, and it is the de facto programming language for building Angular 4 apps.
In Angular 4, we define a component by wrapping a config object with an @Component decorator. We can share code between packages by importing the classes we need; and, in this case, we import Component from the @angular/core package. The LoginComponent class is the component’s controller, and we use the export operator to make it available for other classes to import.
Add the following HTML to the login.component.html:
1 2 3 |
|
Next, configure the routes, via the RouterModule in the app.module.ts file:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
|
Finish enabling routing by replacing all HTML in the app.component.html file with the <router-outlet> tag:
1
|
|
Run ng serve in your terminal, if you haven’t already, and then navigate to http://localhost:4200/login. If all went well you should see the just a test text.
Bootstrap Style
To quickly add some style, update the index.html, adding in Bootstrap and wrapping the <app-root></app-root> in a container:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
You should see the app auto reload as soon as you save.
Auth Service
Next, let’s create a global service to handle a user logging in, logging out, and signing up:
1
|
|
Edit the auth.service.ts so that it has the following code:
1 2 3 4 5 6 7 8 |
|
Remember how providers worked in Angular 1? They were global objects that stored a single state. When the data in a provider changed, any object that had injected that provider would receive the updates. In Angular 4, providers retain their special behavior and they are defined with the @Injectable decorator.
Sanity Check
Before adding anything significant to AuthService, let’s make sure the service itself is wired up correctly. To do that, within login.component.ts inject the service and call the test() method:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
We’ve introduced some new concepts and keywords. The constructor() function is a special method that we use to set up a new instance of a class. The constructor() is where we pass any parameters that the class requires, including any providers (i.e., AuthService) that we want to inject. In TypeScript, we can hide variables from the outside world with the private keyword. Passing a private variable in the constructor is a shortcut to defining it within the class and then assigning the argument’s value to it. Notice how the auth variable is accessible to the this object after it is passed into the constructor.
We implement the OnInit interface to ensure that we explicitly define a ngOnInit() function. Implementing OnInit ensures that our component will be called after the first change detection check. This function is called once when the component first initializes, making it the ideal place to configure data that relies on other Angular classes.
Unlike components, which are automatically added, services have to be manually imported and configured on the @NgModule. So, to get it working, you’ll also have to import the AuthService in app.module.ts and add it to the providers:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
|
Run the server and then navigate to http://localhost:4200/login. You should see working logged to the JavaScript console.
User Login
To handle logging a user in, update the AuthService like so:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
We employ the help of some built-in Angular classes, Headers and Http, to handle our AJAX calls to the server.
Also, update the app.module.ts file to import the HttpModule.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
|
Here, we are using the Http service to send an AJAX request to the /user/login endpoint. This returns a promise object.
NOTE: Make sure to remove
console.log(this.auth.test());from theLoginComponentcomponent.
User Registration
Let’s go ahead and add the ability to register a user as well, which is similar to logging a user in. Update src/app/services/auth.service.ts, taking note of the register method:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
|
Now, to test this we need to set up a back end…
Server-side Setup
For the server-side, we’ll use the finished project from a previous blog post, Token-Based Authentication With Flask. You can view the code from the flask-jwt-auth repository.
NOTE: Feel free to use your own server, just make sure to update the
baseURLin theAuthService.
Clone the project structure in a new terminal window:
1
|
|
Follow the directions in the README to set up the project, making sure the tests pass before moving on. Once done, run the server with python manage.py runserver, which will listen on port 5000.
Sanity Check
To test, update LoginComponent to use the login and register methods from the service:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
|
Refresh http://localhost:4200/login in the browser and you should see a success in the JavaScript console, after the user is logged in, with the token:
1 2 3 4 5 |
|
Auth Login
Update login.component.html:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
Take note of the form. We used the [(ngModel)] directive on each of the form inputs to capture those values in the controller. Also, when the form is submitted, the ngSubmit directive handles the event by firing the onLogin() method.
Now, let’s update the component code, adding in onLogin():
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
|
If you have the Angular web server running, you should see the error Cannot find module '../../models/user' in the browser. Before our code will work, we need to create a User model.
1
|
|
Update src/app/models/user.ts:
1 2 3 |
|
Our User model has two properties, email and password. The ? character is a special operator that indicates that initializing User with explicit email and password values is optional. This is equivalent to the following class in Python:
1 2 3 4 |
|
Don’t forget to update auth.service.ts to use the new object.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
|
One last thing. We need to import the FormsModule in the app.module.ts file.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
|
So, when the form is submitted, we capture the email and password and pass them to the login() method on the service.
Test this out with-
- email:
michael@realpython.com - password:
michael
Again, you should see a success in the javaScript console with the token.
Auth Register
Just like for the login functionality, we need to add a component for registering a user. Start by generating a new Register component:
1
|
|
Update src/app/components/register/register.component.html
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
Then, update src/app/components/register/register.component.ts as follows:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
|
Add a new route handler to the app.module.ts file:
1 2 3 4 |
|
Test it out by registering a new user!
Local Storage
Next, let’s add the token to Local Storage for persistence by replacing the console.log(user.json()); with localStorage.setItem('token', user.data.token); in src/app/components/login/login.component.ts:
1 2 3 4 5 6 7 8 9 |
|
Do the same within src/app/components/register/register.component.ts:
1 2 3 4 5 6 7 8 9 |
|
As long as that token is present, the user can be considered logged in. And, when a user needs to make an AJAX request, that token can be used.
NOTE: Besides the token, you could also add the user id and email to Local Storage. You would just need to update the server-side to send back that info when a user logs in.
Test this out. Ensure that the token is present in Local Storage after you log in.
User Status
To test out login persistence, we can add a new view that verifies that the user is logged in and that the token is valid.
Add the following method to AuthService:
1 2 3 4 5 6 7 8 |
|
Take note of Authorization: 'Bearer ' + token. This is called a Bearer schema, which is sent along with the request. On the server, we are simply checking for the Authorization header, and then whether the token is valid. Can you find this code on the server-side?
Then, generate a new Status component:
1
|
|
Create the HTML template, src/app/components/status/status.component.html:
1 2 3 4 5 6 7 |
|
And change the component code in src/app/components/status/status.component.ts:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
|
Finally, add a new route handler to the app.module.ts file:
1 2 3 4 5 |
|
Ready to test? Log in, and then navigate to http://localhost:4200/status. If there is a token in Local Storage, you should see:
1 2 3 4 |
|
Why? Well, if you dig deeper on the server-side, you will find that the token is only valid for 5 seconds in project/server/models.py:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
|
Update this to 1 day:
1
|
|
And then test it again. You should now see something like:
1 2 3 4 5 6 7 8 9 |
|
Finally, let’s redirect to the status page after a user successfully registers or logs in. Update src/app/components/login/login.component.ts like so:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
|
Then update src/app/components/register/register.component.ts:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
|
Test it out!
Route Restriction
Right now, all routes are open; so, regardless of whether a user is logged in or not, they they can access each route. Certain routes should be restricted if a user is not logged in, while other routes should be restricted if a user is logged in:
/– no restrictions/login– restricted when logged in/register– restricted when logged in/status– restricted when not logged in
To achieve this, add either EnsureAuthenticated or LoginRedirect to each route, depending on whether you want to guide the user to the status view or the login view.
Start by creating two new services:
1 2 |
|
Replace the code in the ensure-authenticated.service.ts file as follows:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
And replace the code in the login-redirect.service.ts like so:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
Finally, update the app.module.ts file to import and configure the new services:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
|
Note how we are adding our services to a new route property, canActivate. The routing system uses the services in the canActivate array to determine whether to display the requested URL path. If the route has LoginRedirect and the user is already logged in, then they will be redirected to the status view. Including the EnsureAuthenticated service redirects the user to the login view if they attempt to access a URL that requires authentication.
Test one last time.
What’s Next?
In this tutorial, we went through the process of adding authentication to an Angular 4 + Flask app using JSON Web Tokens.
What’s next?
Try switching out the Flask back-end for a different web framework, like Django or Bottle, using the following endpoints:
/auth/register/auth/login/auth/logout/auth/user
Add questions and/or comments below. Grab the final code from the angular4-auth repo.