Oracle could leave Java EE to an open source foundation and more news

In this week's edition of our open source news roundup, we take a look at Oracle wanting to pass Java EE to an open source foundation, Schiphol airport turns to open source, Colorado investigating open source textbooks, and more.

Open source news roundup for August 20-September 2, 2017
read more

Audio firm Harman looks to treble annual sales after Samsung takeover

Audio specialist Harman International wants to nearly treble annual sales by 2025 by partnering with tech groups such as Amazon, Google and Microsoft

Microsoft pushes 'mixed reality' features with Windows 10 update

Microsoft is to update its flagship operating system next month so that the latest generation of Windows 10 hardware devices and software can tap into augmented and virtual reality technologies, executives said on Friday.

How to Play Music on Multiple Amazon Echo Speakers (like a Sonos)

Amazon is a little late to the whole-house audio party. Ecosystems like AirPlay and Sonos had them beat for a while, but Amazon has finally added the ability to play music on multiple Echos at once. Read on as we show you how to configure a whole-house system using your Echo speakers.

Catalin George Festila: The beauty of Python: subprocess module - part 4 .

This series of python tutorials that we started at the beginning of this blog and called "The beauty of Python" is part of the series of tutorials aimed at the simplicity and beauty of the python programming language.
The main goal for us is how to use this programming language in everyday life with different tasks.
Today I will come up with examples to cover this goal and show you how to use the subprocess python module.

• using the powershell with python :

• >>> import subprocess
  >>> process=subprocess.Popen(["powershell","Get-Childitem C:\\Windows\\*.log"],stdout=subprocess.PIPE);
  >>> result=process.communicate()[0]
  >>> print result
  

• get and print the hostname :

• >>> print subprocess.check_output("hostname")
  

• print the output of ping command :

• >>> print subprocess.check_output("ping localhost", shell=True)
  

• print the output of dir command :

• >>> cmd = 'dir *'
  >>> supcmd = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  >>> print supcmd.communicate()[0]
  
  

• run the python script like python shell :

• >>> import sys
  >>> import subprocess
  >>> pid = subprocess.Popen([sys.executable, "calc.py"])
  

Peter Bengtsson: Fastest way to match a filename's extension in Python

tl;dr; By a slim margin, the fastest way to check a filename matching a list of extensions is filename.endswith(extensions)

This turned out to be premature optimization. The context is that I want to check if a filename matches the file extension in a list of 6.

The list being ['.sym', '.dl_', '.ex_', '.pd_', '.dbg.gz', '.tar.bz2']. Meaning, it should return True for foo.sym or foo.dbg.gz. But it should return False for bar.exe or bar.gz.

I put together a litte benchmark, ran it a bunch of times and looked at the results. Here are the functions I wrote:

def f1(filename):
    for each in extensions:
        if filename.endswith(each):
            return True
    return False


def f2(filename):
    return filename.endswith(extensions_tuple)


regex = re.compile(r'({})$'.format(
    '|'.join(re.escape(x) for x in extensions)
))


def f3(filename):
    return bool(regex.findall(filename))


def f4(filename):
    return bool(regex.search(filename))

The results are boring. But I guess that's a result too:

FUNCTION             MEDIAN               MEAN
f1 9543 times        0.0110ms             0.0116ms
f2 9523 times        0.0031ms             0.0034ms
f3 9560 times        0.0041ms             0.0045ms
f4 9509 times        0.0041ms             0.0043ms

For a list of ~40,000 realistic filenames (with result True 75% of the time), I ran each function 10 times. So, it means it took on average 0.0116ms to run f1 10 times here on my laptop with Python 3.6.

More premature optimization

Upon looking into the data and thinking about this will be used. If I reorder the list of extensions so the most common one is first, second most common second etc. Then the performance improves a bit for f1 but slows down slightly for f3 and f4.

Conclusion

That .endswith(some_tuple) is neat and it's hair-splittingly faster. But really, this turned out to not make a huge difference in the grand scheme of things. On average it takes less than 0.001ms to do one filename match.

Nest Thermostat E vs. Nest Thermostat: What’s the Difference?

Nest has unveiled it’s latest addition to its smart thermostat lineup, known as the Nest Thermostat E. The original Nest Thermostat is still available and will continue to sell alongside the new model, but what does the Nest Thermostat E bring to the table? Here’s what you need to know.

Geek Trivia: Which State Has The Highest Accuracy In Predicting Presidential Winners?

Think you know the answer? Click through to see if you're right!

How to Change the Appearance of Netflix’s Subtitles

Netflix has decent subtitles, but sometimes they can be hard to read if they appear over a light background, or if your eyesight isn’t what it used to be. Fortunately, you can customize those subtitles to make them look however you want. Here’s how to change their size, font, color, background, and more.

Former Galaxy Note 7 Owners Can Get a Great Deal On a Note 8

The Galaxy Note 7 (no, I’m not going to call it the “Note7” no matter how many times the brand managers email me) was something of a disaster for Samsung. Those fans who put down the better part of a thousand bucks for the flagship phone a year ago were disheartened to learn that their top-of-the-line gadgets had an unusually high chance of melting through their pockets. It was, to put it lightly, a bummer.

How to Take a Screenshot on Almost Any Device

You’re a grownup. You know how to use a computer and a phone. So when it’s time to show off some portion of your screen, don’t try to take a photo of it—that’s kid’s stuff, and it looks like junk anyway. Just about every modern operating system has some method of saving what’s on your screen, and most of them make it pretty easy. Keep this simple guide bookmarked for every method you’ll ever need.

Python Data: Forecasting Time Series data with Prophet – Trend Changepoints

S&P500 Prophet Model with Manually Set Changepoints

In these posts, I’ve been looking at using Prophet to forecast time series data at a monthly level using sales revenue data.  In this post, I want to look at a very interesting aspect of Prophet (and time series analysis) that most people overlook  – that of trend changepoints. This is the fourth in a series of posts about using Prophet to forecast time series data. The other parts can be found here:

• Forecasting Time Series data with Prophet – Part 1
• Forecasting Time Series data with Prophet – Part 2
• Forecasting Time Series data with Prophet – Part 3

Trend changepoint detection isn’t an easy thing to do. You could take the naive approach and just find local maxima and minima but those may or may not be changes in the overall trend of your signal.   There are many different methods for changepoint detection (a good paper looking at four methods can be found here – Trend analysis and change point techniques: a survey) but thankfully Prophet does trend changepoint detection behind the scenes for us (and it does a pretty good job of it).
For most people / tasks, the automatic detection performed by Prophet is good enough, but it never hurts to know how to tweak Prophet in case it misses some changepoints.
I’ve uploaded a jupyter notebook here and the sample data that I’m using here.  Rather than use the monthly sales data I’ve been using, i wanted to use something that has a bit more of a noticable trend so I grabbed the S&P 500 index from FRED.
The jupyter notebook has bit more detail about loading data and running prophet for this data, so I’ll just throw the commands here and you can jump over there to see more detail. These first steps are no different than the standard data loading / prep for prophet discussed in previous posts.

market_df = pd.read_csv('../examples/SP500.csv', index_col='DATE', parse_dates=True)
df = market_df.reset_index().rename(columns={'DATE':'ds', 'SP500':'y'})
df['y'] = np.log(df['y'])

#lets take a look at our data quickly
df.set_index('ds').y.plot()

SP500 Daily Data Plotted
Now, let’s run prophet. Again, this is no different than the steps we’ve taken in previous posts for prophet.

model = Prophet()
model.fit(df);
future = model.make_future_dataframe(periods=366)
forecast = model.predict(future)

Prophet has created our model and fit the data. It has also (behind the scenes) created some potential changepoints. We can access these changepoints with .changepoints.  By default, Prophet adds 25 changepoints into the initial 80% of the data-set. The number of changepoints can be set by using the n_changepoints parameter when initializing prophet (e.g., model=Prophet(n_changepoints=30).
You can view the changepoints by typing the following:

model.changepoints

In addition to viewing the dates of the changepoints, we can also view a chart with changepoints added.

figure = model.plot(forecast)
for changepoint in model.changepoints:
    plt.axvline(changepoint,ls='--', lw=1)

S&P 500 Prophet Model with Changepoints Added (in oragen)
Taking a look at the possible changepoints (drawn in orange/red) in the above chart, we can see they fit pretty well with some of the highs and lows.
Prophet will also let us take a look at the magnitudes of these possible changepoints. You can look at this visualization with the following code:

deltas = model.params['delta'].mean(0)
fig = plt.figure(facecolor='w')
ax = fig.add_subplot(111)
ax.bar(range(len(deltas)), deltas)
ax.grid(True, which='major', c='gray', ls='-', lw=1, alpha=0.2)
ax.set_ylabel('Rate change')
ax.set_xlabel('Potential changepoint')
fig.tight_layout()

SP500 Prophel Model changepoint Magnitudes
We can see from the above chart, that there are quite a few of these changes points (found between 10 and 20 on the chart) that are very minimal in magnitude and are most likely to be ignored by prophet during forecasting be used in the forecasting.
Now, if we know where trends changed in the past, we can add these known changepoints into our dataframe for use by Prophet. For this data, I’m going to use the FRED website to find some of the low points and high points to use as trend changepoints. Note: In actuality, just because there is a low or high doesn’t mean its a real changepoint or trend change, but let’s assume it does.

m = Prophet(changepoints=['2009-03-09', '2010-07-02', '2011-09-26', '2012-03-20', '2010-04-06'])
forecast = m.fit(df).predict(future)
m.plot(forecast);

We can see that by manually setting our changepoints (and only using a few points), we drastically changed the model compared to the model that prophet built for us using the automatic detection of changepoints. Unless you are very sure about your trend changepoints in the past, its probably good to keep the defaults that prophet provides.

Conclusion

Prophet’s use (and accessibility) of trend changepoints is wonderful, especially for those signals / datasets that have significant changes in trend during the lifetime of the signal. That said, unless you are certain about your changepoints, it might be best to let prophet do its thing automatically.
Note:  Please don’t think that because prophet does an OK job of forecasting the SP500 chart historically in this example that you should use it to ‘predict’ the markets. The markets are awfully tough to forecast…I used this market data because I knew there were some very clear changepoints in the data.
The post Forecasting Time Series data with Prophet – Trend Changepoints appeared first on Python Data.

Kushal Das: Introduction to Qubes

I heard the name Qubes OS as an Operating System which was built while keeping security in mind, and also they used Fedora as the base Operating System. A reasonably secure operating system is the tagline and they also have a few testimonials in the site, I guess the most famous one is the following tweet.

The name again came up this week. This time I thought of trying it out, but, this is something I had to do on a bare-metal box, than on a VM. Luckily I bought extra drives in the last PyCon trip. I downloaded the stable 3.2 release, but the installer was failing into a Dracut shell saying /dev/mapper/live-rw is write protected. So, I moved on to the 4.0RC1 image. The installer is a modified Anaconda, means something very familiar to any Fedora/CentOS/Red Hat user. 4.0RC1 is based on Fedora 25, means more known points there.
Qubes uses Xen to manage VMs (for the rest of the post, I will keep using both VM, and domain interchangeably). The dom0 is the initial domain which comes up on boot. It is a short form of Domain 0. This is a privileged domain which manages all the other domains (domU). The default desktop for 4.0rc1 is XFCE. The dom0 does not have standard networking enabled. Actually, to have a working network to connect to outside world, the OS creates two special VMs.

sys-firewall

This special VM is the firewall for every other application VMs. You can actually create more than one firewall VMs and have a different set of rules.

sys-net

This VM has access to the network hardware and can create a connection with your local Wi-Fi or LAN/cable network. In my case, Fedora was failing to create internal interfaces which are being used by sys-firewall, so I rebooted the VM with a Debian-8 template. It solved my networking issue.

Regular Application VMs

When you first install the system, the installer will create a few domains for you, personal, work, untrusted, vault. It will create proper application shortcuts in the menu. This means when you click on the application menu for the Terminal for personal domain, it will first start the domain, and then open up the gnome-terminal for you. Each domain also gets a /rw partition which gets mounted as home. For every reboot, they start from a clean template, with only home consistent between boots.

This also means there is no easy way for applications/malware to talk between different VMs. If you open up a wrong website (with malware) on the untrusted domain, it will not have access to the filesystem under your work domain. There are special ways to copy/paste text between domains.

You can see in the above screenshot, the applications from each domain have different colors, that helps for quick recognition of each application for different domains. You can also see details about the running VMs by clicking on the Qubes icon on the tray in the menubar.

Disposable domains

There are times when you want to open a downloaded file (say PDF or a spreadsheet) on a VM which is only for single use. As soon as you close the application, the VM gets destroyed. For more details, read this document.

VMs without network access

The Vault is a special domain which does not connect to any network. You can also create new VMs in the same way, which does not connect to any firewall VM. The isolation from the network means nothing will go out in case of a malware in the file.
The following screenshot is showing the output of the qvm-ls command in dom0.

I will write more posts in future as I figure out things. Running F26 on the VMs is one them, because Python3.6 :) #qubes channel on Freenode is not that very active, but you will be able to find help in the channel if you wait.

Tips for Horizon Zero Dawn I Learned from My First Playthrough

Horizon Zero Dawn is the best PlayStation game of 2017. I recently finished my first playthrough and have spent a lot of time just thinking about what an incredible game it really is. Let’s talk about it.

Top 5: Your first programming language, running Windows apps on Linux, and more

In this week's top 5, we take a look at open organizations, programming languages, and Linux.

This week's top articles

5. 3 consequences of coding in the open

Erik Kieckhafer shares how transparency has changed the way he works, making him more accountable and more responsive.

read more

How to Remove a Post from a Facebook Group

If someone is posting abusive messages in a Facebook Group you manage, you’ll want to remove it. It’s quick and simple to do, so here’s how.

How to Check Your iPhone’s Battery Health

iOS includes several useful tools for displaying how much battery life your iPhone has left, as well as which apps are consuming the most of your battery. However, none of these tools actually tell you anything about your battery’s long-term health, which is just as important.

Import Python: Python News This Week - EuroSciPy Videos are out, Reducing Python's startup time, Predicting algo ..

Worthy Read EuroSciPy Videos Being uploaded at the time of sending the newsletter. conference , videos Reducing Python's startup time The startup time for the Python interpreter has been discussed by the core developers and others numerous times over the years; optimization efforts are made periodically as well. Startup time can dominate the execution time of command-line programs written in Python, especially if they import a lot of other modules. Python startup time is worse than some other scripting languages and more recent versions of the language are taking more than twice as long to start up when compared to earlier versions (e.g. 3.7 versus 2.7). core-python Python Data Science Handbook - Free To Read This website contains the full text of the Python Data Science Handbook by Jake VanderPlas; the content is available on GitHub in the form of Jupyter notebooks. data science HelloSign eSign API Embed docs directly on your website with a few lines of code. sponsor cachelper Useful cache helpers in one package. caching Predicting Home Prices in Ames, Iowa Regression, Regularization, Residuals and Feature Selection data science Asyncio Coroutine Patterns: Errors and cancellation In the first part of this series we concluded that asyncio is awesome, coroutines are awesome and our code is awesome. But sometimes the outside world is not as awesome and we have to deal with it. Now, for this second part of the series, I’ll run over the options asyncio gives us to handle errors when using these patterns as well as cancelling tasks so as to make our asynchronous systems robust and performant. asyncio Find Top Developers Companies like Airbnb, Pfizer, and Artsy find great developers. Let us find your next great hire. Get started today. sponsor Higher-Level APIs in TensorFlow TensorFlow is providing some higher-level constructs itself, and some new ones were introduced in the latest 1.3 version. In this blog, we’ll look at an example using some of these new higher-level constructs, including Estimator, Experiment, and Dataset. tensorflow Robust Lane Finding Techniques using Computer Vision Lane identification system for camera based systems. machine learning , image processing Python Gems #5: silent function chaining core-python Meme search using pretrained word2vec We show how to build a very basic, yet not bad, meme retrieval system using pretrained word embeddings. machine learning Monitoring Road Traffic with Python machine learning , image processing Listing of Time Series Libraries This curated list contains python packages for time series analysis. time series Jobs Python Engineers, Blockchain at Functional Works Remote Projects setup.py - 1131 Stars, 39 Fork A Human's Guide to setup.py. lolviz - 225 Stars, 7 Fork A simple Python data-structure visualization tool for lists of lists, lists, dictionaries; primarily for use in Jupyter notebooks / presentations. selenium_extensions - 37 Stars, 2 Fork Tools that will make writing tests, bots and scrapers using Selenium much easier. Smoothly-Blend-Image-Patches - 36 Stars, 5 Fork Make smooth predictions by blending image patches, such as for image segmentation cloudflare-partner-cli - 12 Stars, 12 Fork Set CNAME to use Cloudflare using the partner program. janus - 8 Stars, 0 Fork A minimalist argument-parsing library for Python. cbox - 2 Stars, 0 Fork convert any python function to unix-style command.

PyCharm: PyCharm Community Edition and Professional Edition Explained: Licenses and More

We often get questions about the difference between PyCharm Community Edition and PyCharm Professional Edition. We receive further questions about the difference between an individual and a commercial subscription for PyCharm Professional Edition. So let’s try to address some of these questions here:

PyCharm Community Edition

The community edition of PyCharm is Apache 2 licensed: meaning it is free and open source and you can go to GitHub, and look at the source code. You’re free to use it whenever, and wherever you like, including at work. Additionally, you can fork and modify it. See the python subfolder README.md for details about PyCharm rather than IntelliJ IDEA.

What can I use PyCharm Community Edition for?

Let’s go to the LICENSE.txt in the root of the GitHub repo. JetBrains’ open source projects are generally licensed under the Apache 2.0 License. This means that you can use it anywhere you’d like to, and modify it freely. There are some restrictions, which we’ll look into below.

Can I use PyCharm Community Edition at work?

Yes, you can. You are allowed to use PyCharm Community Edition for commercial use.

Can I use PyCharm Community Edition at my university?

Yes, you can. However, you may be interested in learning about our free all product pack licenses for educational usage.

Could I fork PyCharm?

Yes, you can. The Apache 2.0 license doesn’t just permit using this code, but also allows modification. However, before you release YourNamePyCharm, you need to be aware that that the JetBrains and PyCharm trademarks are restricted. So if you do want to fork PyCharm, you will need to take out our branding. Furthermore, in your derived version, you will need to credit us. So you could make ‘YourNameIDE’, with a notice that it’s based on software made by JetBrains.

What can’t I do with PyCharm Community Edition?

There are some restrictions that apply when you fork PyCharm. The Apache 2.0 license requires:

• You need to attribute us. So if you fork PyCharm, you’re not allowed to remove all notices that JetBrains made it
• You need to include a full copy of the Apache 2 license
• You need to include the NOTICES file

We’re not lawyers, so please be aware that this is not legal advice.

So why would I use PyCharm Professional Edition?

The professional edition of PyCharm gives you access to additional features that you don’t get in the community edition:

• Support for Remote Development. PyCharm Pro can deploy and debug python code running on remote machines, virtual machines, and Docker containers.

• Web Development. Django, Flask, and other python frameworks are better supported in PyCharm Pro. Furthermore, HTML, JavaScript, and CSS are only supported in the professional edition. PyCharm Professional edition bundles all features from WebStorm, JetBrains’ JavaScript IDE.

• Database support. PyCharm Professional takes its database support from DataGrip, the SQL IDE by JetBrains. This means you can explore your database within the IDE, and get schema-aware code completion when writing an SQL statement in Python code.

• Some further frameworks and technologies, see our edition comparison page for details.

Alright, so what about Individual v Commercial subscriptions?

Many people get confused when we tell them that they are allowed to use a personal license at work. We believe though, that it’s important that developers can use the tools that are right for the job, and therefore we offer low price options to individual developers.

The difference between personal and commercial licenses isn’t about who uses the software; it is about who owns the software.

• The personal license is yours: you pay, and you own it. You can use it at work, and if you change jobs you can use it at your next job.
• The commercial license is your employer’s: they pay, and they get to keep it if you leave. However, if you buy it and get reimbursed by your employer, you still need a commercial license: if the employer pays, it needs to be a commercial license.

Can I use my license on multiple machines?

Individual licenses: yes. Commercial licenses: yes, as long as your user name (login) is the same on all the machines that you’re running it on.

Can I still use PyCharm Professional Edition after my subscription expires?

If you’ve had a subscription for at least one year: yes. You have a perpetual fallback license for the version that was released one year before your subscription expired, and all its minor updates. See here for details.

Can I get PyCharm Professional Edition for free?

Maybe.

• Are you a student or teacher? Apply for a free student license
• Are you a core contributor or committer for an open source project, no matter the size? As long as you don’t make money from your project, you can likely get a free license to work on your project. Apply for an open source license
• Would you like to have PyCharm on computers in your classrooms? Whether you’re an accredited institute or a commercial provider, we provide free classroom licenses here.
• Do you run a Python user group, and would you like some licenses you can use as prizes in competitions or something similar? Apply for user group support here

We also offer discounts for startups, recent graduates, and users of commercial competitor products, find out more about discounts on our website.

Do you still have a question?

If you have a specific sales question, contact our sales team to learn more about licenses, prices, discounts, etc. If you have any other question, let us know in the comments below, or reach out to us on Twitter.

Diversity and inclusion: Stop talking and do your homework

Open source undoubtedly has a diversity problem. In fact, tech has a diversity problem.

read more

An economically efficient model for open source software license compliance

"The Compliance Industrial Complex" is a term that evokes dystopian imagery of organizations engaging in elaborate and highly expensive processes to comply with open source license terms. As life often imitates art, many organizations engage in this practice, sadly robbing them of the many benefits of the open source model. This article presents an economically efficient approach to open source software license compliance.

Open source licenses generally impose three requirements on a distributor of code licensed from a third party:

read more

Opensource.com CFP and September preview

We're looking for open source-angled articles for a few upcoming themes:

read more

TomTom opens its third traffic centre in Pune

This will showcase the company's intelligent technology for traffic and travel management and how data can be converted to actionable insights

Continuum Analytics Blog: Introducing Anaconda Enterprise 5

From its inception, Anaconda has had the mission to build better products for data scientists and the organizations they serve. We were the first company to launch a collaborative notebook product back in 2012. In this 5-year journey, we’ve learned a lot from our customers and we’ve worked to empower their organizations to effectively leverage …
Read more →

Continuum Analytics Blog: Anaconda Enterprise 5 Introduces Secure Collaboration to Amplify the Impact of Enterprise Data Scientists

Austin, TX—August 31—Anaconda, the Python data science leader, today introduced Anaconda Enterprise 5 software to help organizations respond to customers and stakeholders faster, deliver strategic insight for rapid decision-making and take advantage of cutting edge machine learning. Building on the world’s most popular Python data science platform with over 4.5 million users, Anaconda Enterprise 5 …
Read more →

Dataquest: Machine Learning Fundamentals: Predicting Airbnb Prices

Machine learning is easily one of the biggest buzzwords in tech right now. Over the past three years Google searches for “machine learning” have increased by over 350%. But understanding machine learning can be difficult — you either use pre-built packages that act like ‘black boxes’ where you pass in data and magic comes out the other end, or you have to deal with high level maths and linear algebra.

This tutorial is designed to introduce you to the fundamental concepts of machine learning — you’ll build your very first model from scratch to make predictions, while understanding exactly how your model works.

This tutorial is based on our Dataquest Machine Learning Fundamentals course, which is part of our Data Science Learning Path. The course goes into a lot more detail, and allows you to follow along writing code to learn by doing.

To start though, let’s explore what machine learning actually is.

What is machine learning?

Machine learning is the practice of building systems, known as models, that can be trained using data to find patterns which can then be used to make predictions on new data.

An important distinction is that a machine learning...

Continuum Analytics Blog: Anaconda: A Coming of Age Story

Earlier this week, Continuum Analytics was officially renamed  Anaconda. This change is exciting, and equally as exciting is the journey, from our humble beginnings to today—a community of 4.5 million Anaconda users worldwide. No one tells our story better than Continuum co-founders, Travis Oliphant and Peter Wang. Read on for their thoughts and feelings on …
Read more →

Codementor: Some tricky Python snippets that may bite you off!

A collection of subtle and tricky Python examples

The Best Digital Tools for Dungeons and Dragons

There are many Dungeon and Dragons purists who—even in the digital age—still rely on using old-fashioned tools from earlier decades. However, there are now a ton of digital resources that can enhance D&D greatly, both for players and dungeon masters alike.

Geek Trivia: The Only U.S. State With Commercial Coffee Bean Production Is?

Think you know the answer? Click through to see if you're right!