Data Science, Machine Learning, Natural Language Processing, Text Analysis, Recommendation Engine, R, Python
Saturday, 14 November 2020
Boston Dynamics dog robot 'Spot' learns new tricks on BP oil rig
Working on an oil rig operated by BP Plc nearly 190 miles (305 km) offshore in the Gulf of Mexico, the company is programming Spot to read gauges, look for corrosion, map out the facility and even sniff out methane on its Mad Dog rig.
Adam Ballard, BP's facilities technology manager, said tasks performed by Spot will make the work on the rig safer by reducing the number of people. It also will free up personnel to do other work.
"Several hours a day, several operators will walk the facility; read gauges; listen for noise that doesn't sound right; look out at the horizon for anomalies, boats that may not be caught on radar; look for sheens," Ballard said.
"What we're doing with ...
Read More on Datafloq
Google at odds with U.S. over protective order for firms tied to lawsuit: court filing
Google is pressing for two in-house attorneys to have access to the confidential data while the Justice Department has disagreed, Google said in a court filing on Friday.
In the filing, Google argued it needed the information to prepare an effective defense. It also offered to ensure that any confidential information would be made available solely to two in-house attorneys at the offices of Google's outside counsel or in another secure manner, adding that it would promptly report any disclosure.
The companies, which apart from Microsoft Corp include Oracle Corp, AT&T Inc, Amazon.com, Comcast Corp and others, have until next Friday to make their proposals for the terms ...
Read More on Datafloq
Turkey fines Google $26 million for abusing market position: competition board
The company has been found to be violating the terms of fair competition due to unfair access to advertisement space, the statement said, and the California-based tech giant "was abusing its dominant power in the market".
In February, the competition authority fined Google 98 million lira for abusing its dominant market position and "aggressive competition tactics."
(Reporting by Daren Butler and Ece Toksabay; Editing by Dominic Evans)
...
Read More on Datafloq
North Korean, Russian hackers target COVID-19 researchers: Microsoft
WASHINGTON (Reuters) - Hackers working for the Russian and North Korean governments have targeted more than half a dozen organizations involved in COVID-19 treatment and vaccine research around the globe, Microsoft <MSFT.O> said on Friday.
The software company said a Russian hacking group commonly nicknamed "Fancy Bear" - along with a pair of North Korean actors dubbed "Zinc" and "Cerium" by Microsoft - were implicated in recent attempts to break into the networks of seven pharmaceutical companies and vaccine researchers in Canada, France, India, South Korea, and the United States.
Microsoft said the majority of the targets were organizations that were in the process of testing COVID-19 vaccines. Most of the break-in attempts failed but an unspecified number succeeded, it added.
Few other details were provided by Microsoft. It declined ...
Read More on Datafloq
EU Commission seeks feedback on new data transfer tools after court ruling
BRUSSELS (Reuters) - The European Commission on Friday sought feedback on two new data transfer tools after Europe's top court in July set strict conditions for such mechanisms used by thousands of companies to transfer Europeans' data around the world for various services.
The Luxembourg-based EU Court of Justice upheld the validity of the data transfer mechanism known as standard contractual clauses (SCCs) in a case involving Facebook and Austrian privacy activist Max Schrems, who has campaigned about the risk of U.S. intelligence agencies accessing data on Europeans.
But judges said privacy watchdogs must suspend or prohibit transfers outside the EU if other countries cannot assure that the data will be protected.
The EU executive has since then scrambled to find a solution as companies grapple with the implications and ...
Read More on Datafloq
Nio stock falls after short-seller Citron targets EV maker
Nio's ES6 hatchback model faces imminent threat from likely price cuts for Tesla's Model Y in China, Andrew Left-owned Citron said in an investor note.
Left has long targeted companies that he thinks are over-valued. Friday's take is a reversal to the firm's original recommendation two years ago, when it urged investors to buy the stock.
"Anyone buying NIO stock now is not buying a company or its prospects, rather you are buying 3 letters that move on a screen," Citron said in the note.
Nio did not respond to a request for comment.
...
Read More on Datafloq
Volkswagen boosts investment in electric and autonomous car technology to $86 billon
Under a plan presented on Friday, Volkswagen said it would allocate nearly half its investment budget of 150 billion euros on e-mobility, hybrid cars, a seamless, software-based vehicle operating system and self-driving technologies.
In last year's plan, the German car and truck maker, which owns brands including VW, Audi, Porsche, Seat and Skoda, had earmarked 60 billion euros for electric and self-driving vehicles out of the 150 billion budget.
A global clampdown on emissions, partly triggered by VW's diesel pollution scandal in 2015, has forced carmakers to accelerate the development of low-emission technology, even for their low-margin mainstream models.
...
Read More on Datafloq
Friday, 13 November 2020
MLflow 1.12 Features Extended PyTorch Integration
MLflow 1.12 features include extended PyTorch integration, SHAP model explainability, autologging MLflow entities for supported model flavors, and a number of UI and document improvements. Now available on PyPI and the docs online, you can install this new release with pip install mlflow==1.12.0 as described in the MLflow quickstart guide.
In this blog, we briefly explain the key features, in particular extended PyTorch integration, and how to use them. For a comprehensive list of additional features, changes and bug fixes read the MLflow 1.12 Changelog.
Support for PyTorch Autologging, TorchScript Models and TorchServing
At the PyTorch Developer Day, Facebook’s AI and PyTorch engineering team, in collaboration with Databricks’ MLflow team and community, announced an extended PyTorch and MLflow integration as part of the MLflow release 1.12. This joint engineering investment and integration with MLflow offer PyTorch developers an “end-to-end exploration to production platform for PyTorch.” We briefly cover three areas of integration:
- Autologging for PyTorch models
- Supporting TorchScript models
- Deploying PyTorch models onto TorchServe
Autologging PyTorch pl.LightningModule Models
As part of the universal autologging feature introduced in this release (see autologging section below), you can automatically log (and track) parameters and metrics from PyTorch Lightning models.
Aside from customized entities to log and track, the PyTorch autolog tracking functionality will log the model’s optimizer names and learning rates; metrics like training loss, validation loss, accuracies; and models as artifacts and checkpoints. For early stopping, model checkpoints, early stopping parameters and metrics are logged too. To understand its mechanics and usage, read the PyTorch autologging example.
Converting PyTorch models to TorchScript
TorchScript is a way to create serializable and optimizable models from PyTorch code. As such any MLflow-logged PyTorch model can be converted into a TorchScript, saved and loaded (or deployed to) a high-performance, independent process, where there is no Python dependency. The process entails following steps:
- Create an MLflow Python model
- Compile the model using JIT and convert to TorchScript model
- Log or save the TorchScript model
- Load or deploy the TorchScript model
# Your PyTorch nn.Module or pl.LightningModule
model = Net()
scripted_model = torch.jit.script(model)
…
mlflow.pytorch.log_model(scripted_model, "scripted_model")
model_uri = mlflow.get_artifact_uri("scripted_model")
loaded_model = mlflow.pytorch.load_model(model_uri)
…
For brevity, we have not included all the code here, but you can examine the example code—IrisClassification and MNIST—in the GitHub mlflow/examples/pytorch/torchscript directory.
One thing you can do with a scripted (fitted or logged) model is use the mflow fluent and mlflow.pytorch APIs to access the model and its properties, as shown in the GitHub examples. Another thing you can do with the scripted model is deploy it to a TorchServe server using TorchServer MLflow Plugin.
Deploying PyTorch models with TorchServe MLflow Plugin
TorchServe offers a flexible, easy tool for serving PyTorch models. Through the TorchServe MLflow deployment plugin, you can deploy any MLflow-logged and fitted PyTorch model. This extended integration completes the PyTorch MLOps lifecycle—from developing, tracking and saving to deploying and serving PyTorch models.
For demonstration, two PyTorch examples—BertNewsClassifcation and MNIST—enumerate steps in how you can use the TorchServe MLflow deployment plugin to deploy a PyTorch saved model to an existing TorcheServe server. Any MLflow-logged and fitted PyTorch model can easily be deployed using mlflow deployments commands. For example:
mlflow deployments create -t torchserve -m models:/my_pytorch_model/production -n my_pytorch_model
Once deployed, you can just easily use mlflow deployments predict command for inference.
mlflow deployments predict --name my_pytorch_model --target torchserve --input-path sample.json --output-path output.json.
SHAP API Offers Model Explainability
As more and more machine learning models are deployed in production as part of business applications that offer suggestive hints or make decisive predictions, machine learning engineers are obliged to explain how a model was trained and what features contributed to its output. One common technique used to answer these questions is SHAP (SHapley Additive exPlanations), a theoretical approach to explain an output of any machine learning model.
To that end, this release includes an mlflow.shap module with a single method mlflow.shap.log_explanation() to generate an illustrative figure that can be logged
as a model artifact and inspected in the UI.
import mlflow
# prepare training data
dataset = load_boston()
X = pd.DataFrame(dataset.data[:50, :8], columns=dataset.feature_names[:8])
y = dataset.target[:50]
# train a model
model = LinearRegression()
model.fit(X, y)
# log an explanation
with mlflow.start_run() as run:
mlflow.shap.log_explanation(model.predict, X)
…
You can view the example code in the docs page and try other examples of models with SHAP explanations in the MLflow GitHub mlflow/examples/shap directory.
Autologging Simplifies Tracking Experiments
The mlflow.autolog() method is a universal tracking API that simplifies training code by automatically logging all relevant model entities—parameters, metrics, artifacts such as models and model summaries—with a single call, without the need to explicitly call each separate method to log respective model’s entities.
As a universal single method, under the hood, it detects which supported autologging model flavor is used—in our case scikit-learn—and tracks all its respective entities to log. After the run, when viewed in the MLflow UI, you can inspect all automatically logged entities.
What’s next
Learn more about PyTorch integration at the Data + AI Summit Europe next week, with a keynote from Facebook AI Engineering Director Lin Qiao and a session on Reproducible AI Using PyTorch and MLflow from Facebook’s Geeta Chauhan.
Stay tuned for additional PyTorch and MLflow detailed blogs. For now you can:
- Read MLflow and PyTorch — Where Cutting Edge AI meets MLOps
- Checkout out the PyTorch and MLFlow mlflow/examples/pytorch/
- Examine SHAP GitHub mlflow/examples/shap/
pip install mlflow==1.12.0and have a go at it.
Community Credits
We want to thank the following contributors for updates, doc changes, and contributions to MLflow release 1.12. In particular, we want to thank the Facebook AI and PyTorch engineering team for their extended PyTorch integration contribution and all MLflow community contributors:
Andy Chow, Andrea Kress, Andrew Nitu, Ankit Mathur, Apurva Koti, Arjun DCunha, Avesh Singh, Axel Vivien, Corey Zumar, Fabian Höring, Geeta Chauhan, Harutaka Kawamura, Jean-Denis Lesage, Joseph Berry, Jules S. Damji, Juntai Zheng, Lorenz Walthert, Poruri Sai Rahul, Mark Andersen, Matei Zaharia, Martynov Maxim, Olivier Bondu, Sean Naren, Shrinath Suresh, Siddharth Murching, Sue Ann Hong, Tomas Nykodym, Yitao Li, Zhidong Qu, @abawchen, @cafeal, @bramrodenburg, @danielvdende, @edgan8, @emptalk, @ghisvail, @jgc128 @karthik-77, @kzm4269, @magnus-m, @sbrugman, @simonhessner, @shivp950, @willzhan-db
--
Try Databricks for free. Get started today.
The post MLflow 1.12 Features Extended PyTorch Integration appeared first on Databricks.
Algorithmic Management: What is It (And What’s Next)?
No matter which side of the debate you fall on, it’s clear that the gig economy is here to stay. But with more and more people signing up for these flexible and freelance work arrangements, how can businesses manage them effectively?
Enter “algorithmic management”: the use of algorithms to oversee the efforts of human workers. As algorithmic management becomes more commonplace, it’s important to understand what this practice is, the pros and cons of using it, and what the future holds.
What is algorithmic management?
Algorithmic management, as the name suggests, is the use of computer algorithms and artificial intelligence techniques to manage a team of human employees. By collecting massive quantities of data, in particular data about employee performance, algorithmic management seeks to automate large portions of the managerial decision-making process.
While it’s tough to estimate just how prevalent algorithmic management is, there are a few ...
Read More on Datafloq
4 New Developments in Big Data and Artificial Intelligence That Will Transform the Way Businesses Operate in 2021
When the COVID-19 pandemic struck, some might have thought that artificial intelligence and machine learning were going to lose their momentum. Just the opposite is the case. The pandemic has made it all too clear that machine learning and artificial intelligence need to continue to gain momentum, especially if there are going to be other pandemics in the future.
Artificial intelligence will continue to affect the technologies that change how we live and how we work. It is already impacting several top loud accounting and invoicing tools, software designed to monitor how employees spend their time working, medical technology, cloud invoicing logistics, and so much more. We can only expect that in 2021 we are going to see artificial intelligence and machine learning impact our lives even more. Here are some things we might expect.
1. Increased Surveillance
Facial recognition technology has grown more powerful thanks to computer vision algorithms. Using computers to identify specific individuals as opposed to looking for patterns among groups of people is controversial. However, people are becoming more tolerant of facial recognition and surveillance ...
Read More on Datafloq
Swedish telecoms regulator to appeal court decision on Huawei exclusion
PTS on Monday halted 5G spectrum auctions after a court suspended parts of its earlier decision, in which it followed Britain in banning Huawei equipment from 5G networks, citing national security risks.
The Chinese company had appealed against PTS' decision to exclude it, saying it wanted a court to check if it had been taken according to the law.
"PTS will appeal the administrative court's decision on inhibition to the next instance," the regulator said in a statement on Friday.
The auctions were originally expected to start this week, and would have benefited Nokia and Ericsson as PTS had asked companies taking part to remove Huawei and ZTE ...
Read More on Datafloq
Panasonic appoints company veteran Kusumi as CEO, replacing Tsuga
TOKYO (Reuters) - Japan's Panasonic Corp has appointed its head of automotive business Yuki Kusumi as the company's next chief executive officer, replacing Kazuhiro Tsuga, who was the architect of a partnership with Tesla Inc.
The 55-year-old will take the reins on April 1, Panasonic said on Friday, after a three-decade career at the company which has seen Kusumi lead the automotive component business and the TV operations, much like his predecessor. Tsuga, 63, will become chairman.
The change comes as Panasonic has begun to benefit from a partnership with Tesla, which was central to the incumbent chief's strategy.
Strong sales of Tesla electric vehicles (EV) have allowed Panasonic's battery business to eke out profits this year, following several years of production troubles and delays at the U.S. partner.
...
Read More on Datafloq
Russia's Ozon targets $750 million in IPO as e-commerce booms: sources
MOSCOW (Reuters) - Russian online retailer Ozon plans to raise about $750 million in an initial public offering (IPO) in the United States to help fund its expansion in a rapidly-growing e-commerce market at home, three financial market sources said.
Russia's fragmented e-commerce market is forecast to grow by more than 40% to 2.5 trillion roubles ($32.4 billion) this year, according to Euromonitor research group, and by 10-15% a year over the next five years.
Revenues at Ozon, a Russian version of U.S. e-commerce giant Amazon.com Inc <AMZN.O>, surged as much as 70% in the first nine months of the year, as people switched to online shopping in the coronavirus pandemic.
Ozon was initially aiming for $500 million in the IPO, but has since raised that to $750 million, ...
Read More on Datafloq
Internet can't be Wild West, EU's Breton tells Google CEO Pichai
BRUSSELS (Reuters) - Europe's industry chief Thierry Breton has warned Alphabet CEO Sundar Pichai that he plans to rein in U.S. tech giants via a raft of new rules to curb the excesses of a "Wild West" internet.
Breton issued the warning in a video-conference call with Pichai late on Thursday, according to a statement from the European Commision.
The comments came after a Google internal document outlined a 60-day strategy to counter the European Union's push for tough new tech rules by getting U.S. allies to push back against Breton.
The call was initiated by Google before the document was leaked.
Breton will announce new draft rules known as the Digital Services Act and the ...
Read More on Datafloq
China drafts rules to govern its booming livestreaming sales industry
BEIJING (Reuters) - China's internet watchdog has drafted rules for the first time to regulate the country's livestreaming marketing industry, stepping up scrutiny on e-commerce marketplaces belonging to the likes of tech giant Alibaba Group and JD.Com.
Last week China published draft regulations aimed at preventing anti-monopolistic behaviour by internet platforms which wiped hundreds of billions of dollars off the value of some tech giants including Alibaba and Tencent.
Livestreaming marketing has seen its popularity surge in the last two years among brands like L'Oreal, Nike, Dyson and online shoppers, and most Chinese e-commerce platforms now offer the option to purchase and sell products via livestreaming.
Telegenic hosts sell goods from personal care products to home appliances in real time and top Chinese livestreamers like "lipstick king" ...
Read More on Datafloq
Factbox: List of 31 Chinese companies designated by the U.S. as military-backed
The order could impact some of China's biggest companies. It is designed to deter U.S. investment firms, pension funds and others from buying and selling shares of 31 Chinese companies that were designated by the Defense Department as backed by the Chinese military earlier this year.
Below is a list of those companies based on Department of Defense data found here https://ift.tt/32FbjVI and here https://ift.tt/36qQHBu. Most of them have subsidiaries listed in mainland China and/or Hong Kong.
Aviation Industry Corporation of China
China Aerospace Science and Technology Corp
...
Read More on Datafloq
Zuckerberg defends not suspending ex-Trump aide Bannon from Facebook -recording
PALO ALTO (Reuters) - Facebook <FB.O> Chief Executive Mark Zuckerberg told an all-staff meeting on Thursday that former Trump White House adviser Steve Bannon had not violated enough of the company's policies to justify his suspension when he urged beheading two senior U.S. officials, according to a recording heard by Reuters.
Zuckerberg acknowledged criticism of Facebook by President-elect Joe Biden but said the company shared some of the Biden team's same concerns about social media. He urged employees not to jump to conclusions about how the new administration might approach regulation of social media companies.
Bannon suggested in a video posted on Nov. 5 that FBI Director Christopher Wray and government infectious diseases expert Anthony Fauci should be beheaded, saying they had been disloyal to U.S. President Donald Trump, who last week lost his re-election bid to Biden.
...
Read More on Datafloq
Fired Amazon worker files discrimination lawsuit over pandemic conditions
NEW YORK (Reuters) - A former Amazon.com Inc worker who protested conditions at his New York City fulfillment center sued the retailer on Thursday, accusing it of discrimination for firing him and for putting Black and Hispanic workers at heightened risk of contracting COVID-19.
In a proposed class action filed in Brooklyn federal court, Christian Smalls alleged Amazon failed to provide needed protective gear to its "predominantly minority" workforce, subjecting them to inferior working conditions than its mainly white managers.
Citing a leaked memo from Amazon's general counsel to Chief Executive Jeff Bezos, Smalls also said Amazon fired him after concluding that as a Black man he was a "weak spokesman" for workers.
He also said Amazon tried to drum up public support by making him the "face" of workers criticizing ...
Read More on Datafloq
American Airlines to offer app detailing pandemic-related travel requirements
The app, VeriFLY, by software firm Daon, allows real-time verification of COVID-19 related credentials, such as diagnostic lab test results, and aims to streamline the check-in and verification process at the airport.
"Piloting this new solution is a direct response to our customers' increasing desire to explore more international travel opportunities," President Robert Isom said in a statement.
After verifying that the traveler's data matches the country's requirements, the app displays either a pass or a fail message.
The app will launch for flights from American's hub in Miami to Jamaica.
...
Read More on Datafloq
U.S. senator urges FTC to interview Facebook ex-officials
WASHINGTON (Reuters) - Senator Marsha Blackburn, a Republican and a tough critic of the big tech companies, urged the Federal Trade Commission on Thursday to interview some former employees of Facebook Inc as part of its probe of the social media giant.
Both the FTC and groups of state attorneys general are widely believed to be planning litigation against Facebook for breaking antitrust law.
In her letter to FTC Chairman Joe Simons, Blackburn referred to an FTC deposition of Facebook chief executive Mark Zuckerberg, adding: "While that is promising, I encourage you to also speak to other Facebook executives and engineers who can reveal the company's real agenda. Many of them fear letting Facebook's dominance go unchecked can hold dark consequences for competitors and consumers alike."
Blackburn specifically urged the ...
Read More on Datafloq



