Friday, 31 January 2020

Automating Digital Pathology Image Analysis with Machine Learning on Databricks

With technological advancements in imaging and the availability of new efficient computational tools, digital pathology has taken center stage in both research and diagnostic settings. Whole Slide Imaging (WSI) has been at the center of this transformation, enabling us to rapidly digitize pathology slides into high resolution images. By making slides instantly shareable and analyzable, WSI has already improved reproducibility and enabled enhanced education and remote pathology services.

Today, digitization of entire slides at very high resolution can occur inexpensively in less than a minute. As a result, more and more healthcare and life sciences organizations have acquired massive catalogues of digitized slides. These large datasets can be used to build automated diagnostics with machine learning, which can classify slides—or segments thereof—as expressing a specific phenotype, or directly extract quantitative biomarkers from slides. With the power of machine learning and deep learning thousands of digital slides can be interpreted in a matter of minutes. This presents a huge opportunity to improve the efficiency and effectiveness of pathology departments, clinicians and researchers to diagnose and treat cancer and infectious diseases.

3 Common Challenges Preventing Wider Adoption of Digital Pathology Workflows

While many healthcare and life sciences organizations recognize the potential impact of applying artificial intelligence to whole slide images, implementing an automated slide analysis pipeline remains complex. An operational WSI pipeline must be able to routinely handle a high throughput of digitizer slides at a low cost. We see three common challenges preventing organizations from implementing automated digital pathology workflows with support for data science:

  1. Slow and costly data ingest and engineering pipelines: WSI images are usually very large (typically 0.5–2 GB per slide) and can require extensive image pre-processing.
  2. Trouble scaling deep learning to terabytes of images: Training a deep learning model across a modestly sized dataset with hundreds of WSIs can take days to weeks on a single node. These latences prevent rapid experimentation on large datasets. While latency can be reduced by parallelizing deep learning workloads across multiple nodes, this is an advanced technique that is out of the reach of a typical biological data scientist.
  3. Ensuring reproducibility of the WSI workflow: When it comes to novel insights based on patient data, it is very important to be able to reproduce results. Current solutions are mostly ad-hoc and do not allow efficient ways of keeping track of experiments and versions of the data used during machine learning model training.

In this blog, we discuss how the Databricks Unified Data Analytics Platform can be used to address these challenges and deploy an end-to-end scalable deep learning workflows on WSI image data. We will focus on a workflow that trains an image segmentation model that identifies regions of metastases on a slide. In this example, we will use Apache Spark to parallelize data preparation across our collection of images, use pandas UDF to extract features based on pre-trained models (transfer learning) across many nodes, and MLflow to reproducibly track our model training.

End-to-end Machine Learning on WSI

To demonstrate how to use the Databricks platform to accelerate a WSI data processing pipeline, we will use the Camelyon16 Grand Challenge dataset. This is an open-access dataset of 400 whole slide images in TIFF format from breast cancer tissues to demonstrate our workflows. A subset of the Camelyon16 dataset can be directly accessed from Databricks under /databricks-datasets/med-images/camelyon16/ (AWS | Azure). To train an image classifier to detect regions in a slide that contain cancer metastases, we will run the following three steps, as shown in Figure 1:

  1. Patch Generation: Using coordinates annotated by a pathologist, we crop slide images into equally sized patches. Each image can generate thousands of patches, and is labeled as tumor or normal.
  2. Deep Learning: We use transfer learning to use a pre-trained model to extract features from image patches and then use Apache Spark to train a binary classifier to predict tumor vs. normal patches.
  3. Scoring: We then use the trained model that is logged using MLflow to project a probability heat-map on a given slide.

Similar to the workflow Human Longevity used to preprocess radiology images, we will use Apache Spark to manipulate both our slides and their annotations. For model training, we will start by extracting features using a pre-trained InceptionV3 model from Keras. To this end, we leverage Pandas UDFs to parallelize feature extraction. For more information on this technique see Featurization for Transfer Learning (AWS|Azure). Note that this technique is not specific to InceptionV3 and can be applied to any other pre-trained model.

Figure 1: Implementing an end-to-end solution for training and deployment of a DL model based on WSI data

Image Preprocessing and ETL

Using open source tools such as Automated Slide Analysis Platform, pathologists can navigate WSI images at very high resolution and annotate the slide to mark sites that are clinically relevant. The annotations can be saved as an XML file, with the coordinates of the edges of the polygons containing the site and other information, such as zoom level. To train a model that uses the annotations on a set of ground truth slides, we need to load the list of annotated regions per image, join these regions with our images, and excise the annotated region. Once we have completed this process, we can use our image patches for machine learning.

Figure 2: Visualizing WSI images in Databricks notebooks

Although this workflow commonly uses annotations stored in an XML file, for simplicity, we are using the pre-processed annotations made by the Baidu Research team that built the NCRF classifier on the Camelyon16 dataset. These annotations are stored as CSV encoded text files, which Apache Spark will load into a DataFrame. In the notebook cell below, we load the annotations for both tumor and normal patches, and assign the label 0 to normal slices and 1 to tumor slices. We then union the coordinates and labels into a single DataFrame.

While many SQL-based systems restrict you to built-in operations, Apache Spark has rich support for user defined functions (UDFs). UDFs allow you to call a custom Scala, Java, Python, or R function on data in any Apache Spark DataFrame. In our workflow, we will define a Python UDF that uses the OpenSlide library to excise a given patch from an image. We define a python function that takes the name of the WSI to be processed, the X and Y coordinates of the patch center, and the label for the patch and creates tile that later will be used for training.

Figure 3. Visualizing patches at different zoom levels

We then use the OpenSlide library to load the images from cloud storage, and to slice out the given coordinate range. While OpenSlide doesn’t natively understand how to read data from Amazon S3 or Azure Data Lake Storage, the Databricks File System (DBFS) FUSE layer allows OpenSlide to directly access data stored in these blob stores without any complex code changes. Finally, our function writes the patch back using the DBFS FUSE layer.

It takes approximately 10 minutes for this command to generate ~174000 patches from the Camelyon16 dataset on databricks datasets. Once our command has completed, we can load our patches back up and display them directly in-line in our notebook.

Training a tumor/normal pathology classifier using transfer learning and MLFlow

In the previous step, we generated patches and associated metadata, and stored generated image tiles using cloud storage. Now, we are ready to train a binary classifier to predict whether a segment of a slide contains a tumor metastasis. To do this, we will use transfer learning to extract features from each patch using a pre-trained deep neural network and then use sparkml for the classification task. This technique frequently outperforms training from scratch for many image processing applications. We will start with the InceptionV3 architecture, using pre-trained weights from Keras.

Apache Spark’s DataFrames provide a built-in Image schema and we can directly load all patches into a DataFrame. We then use Pandas UDFs to transform the images into features based on InceptionV3 using Keras. Once we have featurized each image, we use spark.ml to fit a logistic regression between the features and the label for each patch. We log the logistic regression model with MLFlow so that we can access the model later for serving.

When running ML workflows on Databricks, users can take advantage of managed MLFlow. With every run of the notebook and every training round, MLFlow automatically logs parameters, metrics and any specified artifact. In addition, it stores the trained model that can later be used for predicting labels on data. We refer interested readers to these docs for more information on how MLFlow can be leveraged to manage a full-cycle of ML workflow on databricks.

Table 1 shows the time spent on different parts of the workflow. We notice that the model training on ~170K samples takes less than 25 minutes with an accuracy of 87%.

 Workflow  Time
 Patch generation  10 min
 Feature Engineering and Training  25 min
 Scoring (per single slide)  15 sec

Table 1: Runtime for different steps of the workflow using 2-10 r4.4xlarge workers using Databricks ML Runtime 6.2, on 170,000 patches extracted from slides included in databricks-datasets

Since there can be many more patches in practice, using deep neural networks for classification can significantly improve accuracy. In such cases, we can use distributed training techniques to scale the training process. On the Databricks platform, we have packaged up the HorovodRunner toolkit which distributes the training task across a large cluster with very minor modifications to your ML code. This blog post provides a great background on how to scale ML workflows on databricks.

Inference

Now that we have trained the classifier, we will use the classifier to project a heatmap of probability of metastasis on a slide. To do so, first we apply a grid over the segment of interest on the slide and then we generate patches—similar to the training process—to get the data into a Spark DataFrame that can be used for prediction. We then use MLflow to load the trained model, which can then be applied as a transformation to the DdataFframe which computes predictions.

To reconstruct the image, we use python’s PIL library to modify each tile color according to the probability of containing metastatic sites and patch all tiles together. Figure 4 below shows the result of projecting probabilities on one of the tumor segments. Note that the density of red indicates high probability of metastasis on the slide.

Figure 4: Mapping predictions to a given segment of a WSI

Get Started with Machine Learning on Pathology Images

In this blog, we showed how Databricks along with Spark SQL, SparkML and MLflow, can be used to build a scalable and reproducible framework for machine learning on pathology images. More specifically, we used transfer learning at scale to train a classifier to predict probability that a segment of a slide contains cancer cells, and then used the trained model to detect and map cancerous growths on a given slide.

To get started, sign-up for a free Databricks trial and experiment with the WSI Image Segmentation notebook: Visit our healthcare and life sciences pages to learn about our other solutions.

--

Try Databricks for free. Get started today.

The post Automating Digital Pathology Image Analysis with Machine Learning on Databricks appeared first on Databricks.

Understand the fundamentals of Delta Lake Concept

You might be hearing a lot about Delta Lake nowadays. Yes, it is because of it’s introduction of new features which was not there in Apache Spark earlier. Why is Delta Lake?If you check here, you can understand that Spark writes are not atomic and data consistency is not guaranteed. When metadata itself becomes big data it is difficult to manage data. If you have worked on lambda architecture you would understand how painful it is to have same aggregations to both hot layer as well as cold layer differently. Sometimes you make unwanted writes to a table or a location and it overwrites existing data then you wish to go back to your previous state of data which was tedious task. What is Delta Lake?Delta Lake is a project that was developed by Databricks and now open sourced with the Linux Foundation project. Delta Lake is an open source storage layer that sits on top of the Apache Spark services which brings reliability to data lakes. Delta Lake provides new features which includes ACID transactions, scalable metadata handling, and unifies streaming and batch data processing. Delta Lake runs on top of your existing data lake and is ...


Read More on Datafloq

How Purpose-Driven Tokenisation Will Enable Innovative Ecosystems

Tokens have been around for 1000s of years, but only recently have we seen the rise of digital tokens. Now, cryptographic tokens offer us an opportunity to redesign value streams and hence existing ecosystems. A well-designed token ecosystem unlocks value by bringing parties together in new ways and stimulates the target behaviour by having cryptographic tokens as built-in incentives. Tokens matter and offer us a chance to redesign existing and new ecosystems. 

On January 14, 2020, we had the second round table session organised by the 2Tokens initiative. The 2Tokens project aims to clarify the path to realising value from tokenisation. During the first round table session, we discussed why we need tokenisation, what is required to achieve value from tokenisation, and how we should move ahead with it.

The objective of the second round table discussion was to understand the challenges faced when designing new, token-driven, ecosystems; what is needed to enable purpose-driven tokenisation, which comes down to token engineering - the practice of using tokens as the foundation for designing value flows and ultimately economic systems?

The event took place at YES!Delft and with over 70 thought leaders, innovation drivers and representatives from enterprises, law firms, the regulator and the Dutch ...


Read More on Datafloq

What Difference Will 2020 Bring for a Data Scientist

Data science, the term still sounds fancy in today’s tech industry. However, in reality, it has been around this universe for nearly a hundred years now. In the 1970s, Bayern's theorem taught that having an initial belief with the help of newer data would lead toward a path of improved belief. A career in data science would be an ideal career choice in the job market. It is said that between the years 2011 to 2012, job postings for data scientists increased to 15,000%. The job role of a data scientist is to use the data, analyze it by using different tools and technologies turning these big data into actionable insights. Though it sounds simple, the task and responsibilities that these data scientists take up are challenging. For someone looking to make a career in data and analytics, these are the job roles that are currently in-demand – data scientists, citizen data scientists, data engineers, and AI hardware specialists. Here’s how data science will look in 2020: - AI and machine learning to serve as assistants for a data scientistSounds interesting, doesn’t it?Adopting AI and machine learning does not mean a data science professional will be performing less work. Instead, ...


Read More on Datafloq

Five Ways Big Data is Invaluable to Launching an Online Business

Big data is incredibly valuable for businesses. The market is expected to reach $274 billion in the next two years. Online businesses in particular stand to benefit from big data, due to the logistics of their business models.

Every entrepreneur has some sort of online presence these days since most of us are on social media or have our own website. The good news is that big data is making it easier to create a successful business model.

It’s become so easy for people to host and share their ideas online that having your own website is now actually pretty common. It doesn’t have to be the full-time gig it used to be - a lot of people run them alongside their day jobs, whether it’s for a hobby or even a means to make an easy side hustle. You can make things even easier by using big data to automate many aspects of your business.

Anyone can make a website but if you’re looking to make it profitable or make it the basis for your online business, you’ll want to take it more seriously than just a project to chip away at in your spare time. Running a successful online business requires ...


Read More on Datafloq

Data Lineage - How the History of Your Data can Influence the Future of Your Business

Almost every single industry has been revolutionized to at least some degree by data processing. Whether you manage a local restaurant or a massive distributorship, there's a good chance that you've recently made several decisions based on the availability of data that you previously lacked.Fewer people are paying attention to where their data is coming from, however. In fact, business owners who rely on databases are likely to accept everything they read as the truth. Unfortunately, they don't realize that there are certain coding tolerances involved with collecting information on any digital device.In some cases, an insecure system can actually suffer from bogus information as well. That makes data lineage extremely important to ensure that you are getting the most accurate numbers possible.Implementing a Data Lineage PlanQuite a few people are probably asking the question "what is Data Lineage" because, at the moment, few organizations are investing much into this field. That's quite a shame considering how much promise it shows for those who need to check the quality of the numbers they're getting.Data lineage is essentially the life cycle that every piece of digital information you collect goes through. This includes where the data comes from, how it got ...


Read More on Datafloq

Data Science: Breaking Down The Data Silos

In today’s digitized economy, the capability to utilize data represents a real and indispensable competitive advantage. Organizations are using advanced technologies to unbolt the true value of their data. This allows them to make smart decisions, accelerate innovation, serve up better customer experiences, and respond to problems faster.

The main issue standing in the way of transforming the deluge of big data into decision-making is- Data Silos. Presently, every department has its data repository which is isolated from the rest of the organization, resulting in - many conflicting versions of the truth. Silos have made it difficult to analyze, manage, & activate data. So, here we are to discuss how you can break down the Silos and activate the power of Business.

What are Data Silos?

Data silo refers to independent sets of data within an organization. Often aligned to either IT systems or business functions, data silos are where only a limited group of people have knowledge or access to the data resources available. It is a big problem for organizations that are looking out for sheer productivity, business agility, and efficiency.

Breaking down your data silos starts with knowing how they were initially created? Eliminating data silos can help you get the ...


Read More on Datafloq

Analytics and Artificial Intelligence – A Blue Ocean of Opportunities

“A whole new world …” the lyrics go as Aladdin flies up in the skies on his magical carpet. Yes, a childhood nostalgia, but yes today it can become a business reality. Yes, of course, with the winning combination of Analytics and Artificial Intelligence (AI).

Any business today, you name it, encounters a highly competitive landscape. However, the data that it generates, both structured and unstructured, has a kind of a digital footprint, which can be thoughtfully analyzed and utilized. Though, you need that vantage point to correlate the dots and create the bigger picture. It is the same bird’s eye view, which the above lyrics hint at and allows you to join the dots to identify those blue ocean opportunities.

How does this technology combination work?

Businesses typically have their own enterprise data management platform to manage, store, and retrieve their multi-structured data. They can use Analytics and AI on top of it to correlate their data sets and come up with evolving business trends and patterns. This AI-powered Analytics offers a practical business solution to what otherwise takes months of tedious, manual, and time-consuming analysis.

This technology amalgamation creates a 360-degree analysis of data points along with interactive visualizations, which you can ...


Read More on Datafloq

What Difference Will 2020 Bring for a Data Scientist

Data science, the term still sounds fancy in today’s tech industry. However, in reality, it has been around this universe for nearly a hundred years now. In the 1970s, Bayern's theorem taught that having an initial belief with the help of newer data would lead toward a path of improved belief. A career in data science would be an ideal career choice in the job market. It is said that between the years 2011 to 2012, job postings for data scientists increased to 15,000%. The job role of a data scientist is to use the data, analyze it by using different tools and technologies turning these big data into actionable insights. Though it sounds simple, the task and responsibilities that these data scientists take up are challenging. For someone looking to make a career in data and analytics, these are the job roles that are currently in-demand – data scientists, citizen data scientists, data engineers, and AI hardware specialists. Here’s how data science will look in 2020: - AI and machine learning to serve as assistants for a data scientistSounds interesting, doesn’t it?Adopting AI and machine learning does not mean a data science professional will be performing less work. Instead, ...


Read More on Datafloq

Small is secure in cyber space

Big firms are turning to cyber security startups dotting India as they look for niche solutions as against end-to-end management offered by transnationals

Apple, Broadcom ordered to pay $1.1 billion for patent infringement

Apple was ordered to pay $837 million and Broadcom must pay $270 million to the California Institute of Technology.

AI Initiatives in Manufacturing Often Loosely Defined, Survey Finds

By AI Trends Staff

Many AI initiatives are loosely defined, lack proper technology and data infrastructure, and are often failing to meet expectations, according to a new report from Plutoshift on implementation of AI by manufacturing companies.

A supplier of an AI solution for performance monitoring, Plutoshift surveyed 250 manufacturing professionals with visibility into their company’s AI programs. Overall, the survey found that manufacturing companies are gaining experience while taking a measured approach to implementing AI.

Among the specific findings:

  • 61% said their company has good intentions but needs to reevaluate the way it implements AI projects;
  • 17% said their company was in full implementation stage of their AI project;
  • 84% are not yet able to automatically and continuously act on their data intelligence, while some are gathering data;
  • 72% said it took more time than anticipated for their company to implement the technical/data collection infrastructure needed to take advantage of AI
  • Only 57% said their company implemented AI projects with a clear goal, while almost 20% implemented AI initiatives due to industry or peer pressure to utilize the technology.
  • 17% said their company implemented AI projects because their company felt pressure to utilize this technology from the industry
  • 60% said their company struggled to come to a consensus on a focused, practical strategy for implementing AI

Among its conclusions, the report stated, “To truly utilize data, manufacturing companies need a data infrastructure and platform that is designed around performance monitoring for the physical world. That means gaining the ability to take data from any point in the workflow, analyze that data, and provide reliable predictions at any point. Right now, few companies report these full capabilities and would rethink their direction.”

Plutoshift CEO and Founder Prateek Joshi stated in a  press release about the survey, “Companies are forging ahead with the adoption of AI at an enterprise level. Despite the progress, the reality that’s often underreported is that AI initiatives are loosely defined. Companies in the middle of this transformation usually lack the proper technology and data infrastructure. In the end, these implementations can fail to meet expectations. The insights in this report show us that companies would strongly benefit by taking a more measured and grounded approach toward implementing AI.”

Prateek Joshi, CEO and Founder, Plutoshift

Biggest Players Investing and Gaining Valuable Experience with AI

Another way to gauge how AI is or will penetrate manufacturing is to examine what the biggest players are doing. Siemens, GE, FANUC, and KUKA are all making significant investments in machine learning-powered approaches to improve manufacturing, described in a recent account in emerj. They are using AI to bring down labor costs, reduce product defects, shorten unplanned downtimes and increase production speed.

These giants are using the tools they are developing in their own manufacturing processes, making them the developer, test case, and first customers for many advances.

The German conglomerate, Siemens, has been using neural networks to monitor its steel plants and improve efficiencies for decades. The company claims to have invested around $10 billion in US software companies (via acquisitions) over the past decade. In March of 2016, Siemens launched Mindsphere, described as an “IoT operating system,” and a competitor to GE’s Predix product. Siemens describes Mindsphere as a smart cloud for industry, being able to monitor machine fleets throughout the world. In 2016, it integrated IBM Watson Analytics into its tools service.

Siemens describes an AI success story with its effort to improve gas turbine emissions. “After experts had done their best to optimize the turbine’s nitrous oxide emissions,” stated Dr. Norbert Gaus, Head of Research in Digitalization and Automation at Siemens Corporate Technology, “our AI system was able to reduce emissions by an additional ten to fifteen percent.”

Siemens envisions incorporating its AI expertise within Click2Make, its production-as-a-service technology. It was described in an account in Fast Company in 2017 as a “self-configuring factory.”  Siemens envisions a market where companies submit designs and factories with the facilities and time and handle the order would start an automatic bidding process. The manufacturer would be able to respond with the factory configuring itself. That’s the idea.

Dr. Norbert Gaus, Head of Research in Digitalization and Automation, Siemens Corporate Technology

GE’s Manufacturing Software Strategy a Work in Progress

GE, which has had fits and starts with its software strategy, has over 500 factories worldwide that it is transforming into smart facilities. GE launched its Brilliant Manufacturing Suite for customers in 2015. The first “Brilliant Factory” was built that year in Pune, India, with a $200 million investment. GE claims it improved equipment effectiveness by 18%.

Last year, GE sold off most of the assets of its Predix unit. An account in Medium described reasons for the retrenchment, including a decision to build a Predix cloud data center, and not recognize the competition from Amazon, Microsoft, and Google. Another criticism was that Predix was not known to be developer-friendly. Successful platforms need developer content, and developers need support from a community.

GE’s software strategy in manufacturing is a work in progress.

FANUC Has Invested in AI

FANUC, the Japanese company producing industrial robotics, has made substantial investments in AI. In 2015, Fanuc acquired a stake in the AI startup Preferred Networks, to integrate deep learning into its robots.

In early 2016, FANUC announced a collaboration with Cisco and Rockwell Automation to develop and deploy FIELD (FANUC Intelligent Edge Link and Drive). This was described as an industrial IoT platform for manufacturing. Just a few months later,  with NVIDIA to use their AI chips for their “the factories of the future.”partnered with NVIDIA to use their AI chips for their “the factories of the future.”

FANUC is using deep reinforcement learning to help some of its industrial robots . They perform the same task over and over again, learning each time until they achieve sufficient accuracy. By partnering with NVIDIA, the goal is for multiple robots can learn together. The idea is that what could take one robot eight hours to learn, eight robots can learn in one hour. Fast learning means less downtime and the ability to handle more varied products at the same factory.train themselves. They perform the same task over and over again, learning each time until they achieve sufficient accuracy. By partnering with NVIDIA, the goal is for multiple robots can learn together. The idea is that what could take one robot eight hours to learn, eight robots can learn in one hour. Fast learning means less downtime and the ability to handle more varied products at the same factory.

KUKA Working on Human-Robot Collaboration

KUKA, the Chinese-owned, Germany-based manufacturer of industrial robots, is investing in human-robot collaboration. The company has developed a robot that can work beside a human safely, owing to its intelligent controls and high-performance sensors. KUKA uses them; BMW is also a customer.

Robots that can work safely with humans will be able to be deployed in factories for new tasks, improving efficiency and flexibility.

Read the Plutosoft manufacturing study press release; read the source articles in  emerj,  Fast Company and Medium.

Federal Government Adoption of AI and RPA Spreading; Bots Coming to GSA

By AI Trends Staff

Surround the legacy platforms with a wrapper of automated processes, produced with a combination of AI and RPA — in part to avoid the expense of replacing the legacy platform — is an approach being widely adopted in the federal government today.

RPA is a form of business process automation that “employs” software robots, increasingly imbued with more AI, to do work. The government is also now pursuing Intelligent Process Automation, the application of AI and other technologies — such as computer vision, cognitive automation and machine learning — to RPA.

“Many civilian and federal agencies’ use of information processing, process improvement, and intelligent character recognition have led to the use of AI in robotic process automation (RPA),” stated Anil Cheriyan is Director/Deputy Commissioner, Technology Transformation Services for the US Federal Government, in a recent account in the Enterprisers Project. “We see a significant opportunity to use AI and RPA to automate processes around antiquated systems without the expense associated with replacing the legacy platforms.”

Anil Cheriyan, Director/Deputy Commissioner, Technology Transformation Services, US Federal Government

Cheriyan has experience with AI, having led the digital transformation of SunTrust Banks as CIO. He pursued APIs, robotics, data lakes and cloud computing to help make the bank more efficient. He also worked for IBM Global Business Services, mainly for clients in financial services. Cheriyan earned his Master of Science and Master of Philosophy degrees in Management as well as a Bachelor of Science in Electronic and Electrical Engineering from Imperial College in London, UK.

“AI is a capability that the country needs. Not only is it instrumental in improving the experience and effectiveness of citizens’ engagement with federal services, it also enables core capabilities that strengthen our national security and defense,” he stated.

The GSA has a three-phase framework for moving to RPA and IPA. The first is the evaluation phase, an examination of the end-to-end processes in an agency to determine where “pain points” exist. The RPA/IPA team thinks about whether the process need to be automated or eliminated. Second, process automation tools are employed to implement bots. In a third phase, the bots are monitored and iterated using the automation tools.

“The benefit of using an RPA approach is that you’re not completely replacing the legacy platform, you’re building a layer on top of it to enable automation across those legacy platforms,” stated Cheriyan. “That’s what’s attractive about RPA: Rather than spending five plus years replacing a legacy platform, you can build process automation across legacy platforms using RPA techniques in just a few months.”

Robotics Process Automation Community of Interest in DC

A Robotic Process Automation Community of Interest holds periodic meetings in Washington, to share experiences and challenges. A meeting last fall was hosted by teams from the IRS, the General Services Administration and the Office of Personnel Management.

The IRS could cite six distinct use cases for RPA, according to IRS Deputy Chief Procurement Officer Harrison Smith, who spoke with reporters after the event, according to an account in Nextgov. Smith plans to apply RPA in projects as part of the Pilot IRS program. He is seeing that the automation efforts will not be uniform across the government.

Harrison Smith, Deputy Chief Procurement Officer, IRS

“They’re not all going to look the same,” he stated. “You have to make sure that if it’s an automation solution for another environment, that you have the technology [people] and you have the systems integrators who are able to talk to the people who are actually performing the work.

He encouraged other federal technology managers to engage in a dialogue with the RPA and IPA tool and solution providers, to plug for their own agency’s needs to be addressed currently and in the future. Smith notes that project spending on automation tools is expected to triple in the next two to three years. The current requirements of the IRS are likely to differ from longer-term requirements. “We need to keep those lines of conversation open and moving ahead—making sure everybody is on a similar sheet of music,” he stated.

Antworks CEO Issues Cautions

Government customers of Antworks are using the company’s intelligent automation platform to pursue projects including call center optimization, passport verification and management, records management and vendor onboarding, said Asheesh Mehra, CEO and Co-Founder of Antworks, in response to a query from AI Trends.

He added, “But with AI’s great power comes great responsibility. And government must work alongside business to enable Ethical AI by ensuring people use AI engines as intended and not for fraudulent or malicious purposes. I believe government should create and enforce rules for AI at the application level – defining which applications of AI are acceptable and which are not.

Asheesh Mehra, CEO and Co-Founder Antworks

“Also, government is a major employer. So, it needs to prepare for AI’s impact on its workforce. Government agencies can do that by giving their workers the opportunity not just to upskill, but also to reskill, so they can undertake higher value roles as well as entirely new jobs.”

Strong Growth for RPA Seen by New Forrester Study

A new study from Forrester commissioned by UiPath, supplier of RPA software, to gauge the interest in RPA from business is projecting strong growth.

In report highlights:

  • Investment in automation will rise: 66% of companies stated they plan to increase RPA software spend by at least 5% over the next 12 months
  • Automation will affect roles in different ways: By 2030, some jobs will be cannibalized, some will be created, others will be transformed – but only a few will remain untouched
  • The digital skills gap is a concern for all employees: 41% of respondents say their employees are concerned that their existing digital skills may not match what their job will require in the future
  • Automation education in the workplace will boost career prospects: Training employees, providing them vocational courses, or encouraging them to pursue digital qualifications allows them to overcome fears around automation and embrace it as a productivity-boosting asset.

(To accompany the study, UiPath will host a webinar on Thursday, February 6 at 9 a.m. ET/2 p.m. GMT that will go in depth about the report findings.)

Read the source articles in the Enterprisers Project and in Nextgov. Learn more at UiPath and Antworks.

Dementia Drivers and AI Autonomous Cars

By Lance Eliot, the AI Trends Insider

Do you know someone that seems to be progressively forgetting things and their mind cannot remain focused on matters at-hand?

I’m not referring to the occasional moment whereby you might get distracted and misremember where you left your keys or where you put the TV remote.

We’ve likely all had those moments.

I knew a friend in college that every time he noticed that someone else had lost something or misplaced an item, he would jump right away to the classic “have you lost your mind” and seemed to overplay the rather hackneyed phrase (it became an ongoing irritant to those of us that interacted with him regularly). It is easy to leap to foregone conclusions and falsely suggest that someone has a systemic mental failing.

Typically, regrettably, as we get older, humans do though tend to genuinely have a kind of mental decay and their brains sadly begin to deteriorate.

There are an estimated 5 million people in the United States that are currently experiencing dementia.

Keep in mind that dementia is not a disease per se, though some assume it is, and instead it is considered an umbrella term that encompasses the loss of our thinking skills and also the degradation of various memory processing aspects. Dementia might start with no especially notable impairment and thus not be readily detectable and be easily shrugged off as inconsequential. Gradually, dementia usually emerges as an increasingly persistent onset, which might then ultimately lead to becoming quite severe and debilitating for the person.

This decreasing capability of cognitive functioning can be tough for the person with it and also be tremendously trying for those that are around or connected with the person. Many people that experience dementia are quick to deny they have anything wrong with them at all. It can be excruciatingly embarrassing and frightening to consider that you might have dementia. Some will do more than simply deny they have it and will attempt to showcase that they clearly do not suffer from it. In this effort to disprove the dementia, it often brings even more light to the dementia and perhaps illuminates it more so than others thought existed for the person.

Touching Story Of Dementia

I am reminded of the grandfather of a close friend of mine.

My friend was grappling with his aging grandfather’s behavior and actions that appeared to be symptomatic of dementia. The grandfather would get confused about the days and times that he was supposed to be taking medication for an ailment he had. He would forget the names of loved ones and could not identify their names when they came to visit him. I recall one time that I went to visit him, he brought me a cup of tea, and moments later he asked me if I wanted some tea. I pointed out that I already had tea. Nonetheless, he went back to the kitchen and brought me another cup of tea.

Those kinds of cognitive failings were perhaps reasonably acceptable in the sense that they weren’t preventing him from carrying on day-to-day and living a relatively normal existence. When the symptoms first began, my friend had “the talk” with him about dementia, which I’d say is more awkward than “the talk” of a father telling his son about the birds-and-the-bees. Having a son tell his own father that dementia is taking hold, well, it’s something no one welcomes and likely everyone dreads.

Unfortunately, the dementia oozed into all other aspects of the grandfather’s activities. Of which, the one that had perhaps had the most danger associated with it involved driving a car. The grandfather still had a legal driver’s license. There was nothing legally that prevented him from driving a car. He owned a car. He had the keys to the car. He could freely use the car whenever he wished to do so. Indeed, he tied much of his sense of being to the use of the car. It was his path to freedom. He could drive to the store, or drive to the park, or drive any darned place that he wanted to get to.

I was over at the house with my friend when one day his grandfather announced that he was going for a drive. We watched as he slowly, very slowly, agonizingly slowly, backed out of the garage. As he did so, he also bumped into a child’s bike that was stored in the garage. Furthermore, he was turning the steering wheel as he backed out, which made no sense since the driveway was straight behind the car. He managed to get the car almost turned kitty-corner and it looked like he might drive onto the grass of the front yard. He barely corrected in time and ended-up slightly going off the curb, rather than the usual driveway cut that was amply provided.

He then backed further into the street, doing so at a pace that caused other oncoming cars to come to a halt and wait. It wasn’t just a few brief seconds. It was somewhere around 30 seconds before he was able to fully get into the street, finally taking the car out of reverse, and put it into forward gear, and then eased down the road. I noticed that a neighbor’s dog was off its leash and running around, including veering into the street. I don’t believe the grandfather noticed the dog at all, and the car made no attempts to evade hitting the dog (luckily, the dog scampered on its own back to the grassy yards of the nearby homes).

If you are thinking why I am seemingly criticizing the grandfather about his driving, I’d like to emphasize that it is only because his driving skills had degraded and he was now becoming a danger to himself and others. I fully understand the importance he placed on personal mobility of having a car, along with the control, the emotional boost of driving, and so on. At some point we need to be equally thoughtful about the risk that his driving presents to his own well-being, and the well-being of others that come in contact with his car while he is driving.

Suppose he had hit that dog that was in the street? Suppose when backing out of the garage he had crushed the child’s bike? Suppose as he cut across the grass toward the curb that a small child was there and got struck? Suppose that as he entered into the street, an ongoing car zipped along but he backed into it and caused a car accident. In all of these instances, he could have been injured or killed. He could have injured or killed others. He could have caused damage to property. Etc.

In addition to his memory loses and his cognitive processing loses, he was quite slow to mentally process things.

As you know, when driving a car, you are often confronted with situations that require a split-second kind of mental processing. Is that car going to run the red light, and if so, should you try to do an emergency braking or instead attempt to push on the gas and accelerate out of the situation? In his dementia, it was relatively apparent that he would not be able to make such decisions in the split seconds of time required. This further made his driving a kind of “menace” to the road (I hate to say it that way, but, we need to be honest about these matters, for safety’s sake).

In all fairness, I also stipulate that sometimes there are situations wherein caring people inadvertently ascribe dementia to someone and their driving, when it has no such merit. My own now-adult-driver daughter still believes that I drive too slowly and conservatively.

Do I have dementia? I don’t think so, and nor does she assert it. But the point being that there are different kinds of driving styles and someone might have a different style that others don’t like, but if it is still a fully lucid form of driving and one that exercises due safety and care, let’s not just tarnish it with the dementia brush, so to speak.

 

For more about driving styles, see my article: https://aitrends.com/selfdrivingcars/driving-styles-and-ai-self-driving-cars/

For aspects about the elderly and cars, see my article: https://aitrends.com/ethics-and-social-issues/elderly-boon-bust-self-driving-cars/

For the tit-for-tat involved in driving, see my article: https://aitrends.com/selfdrivingcars/tit-for-tat-and-ai-self-driving-cars/

Why greed is an essential element of driving, see my article: https://aitrends.com/selfdrivingcars/selfishness-self-driving-cars-ai-greed-good/

Driving And Dementia

In general, I would guess that we would all reasonably agree that if someone is hampered by dementia and it does so to the degree that it materially impairs their driving, the person ought to be reconsidering whether they should be driving or not.

I realize that in the case of the grandfather that he still actually had his driver’s license, which you might insist “proves” that he can still sufficiently drive a car. Not really. It was instead more due to a formality in the sense that his driver’s license had not come due for renewal involving a road-level driving test. Instead, he was just paying it for renewal year after year as a paperwork matter. This meant that the Department of Motor Vehicles (DMV) had no ready means to know that the grandfather was now an “impaired” driver.

I suppose you could say that if he was such a bad driver that he would have gotten a traffic ticket. And, if he had gotten a traffic ticket, the police would notify the DMV. Once the DMV was notified, certainly they would formally revoke his driver’s license. Well, he drove just a few miles a couple of times a week and had done so in a town-like area that allowed his poor driving to not stick-out, and thus he didn’t have any tickets as yet.

Would you prefer to wait until he actually hits someone or something, before we raise a red flag? I’d say that’s trying to close the barn door after the horse has already gotten out.

Imagine how someone else would feel if they knew that you knew that the grandfather was unfit to drive a car, and yet the grandfather rammed into them or their children? Why didn’t you take steps to prevent this from happening? The kindness of letting someone with dementia to continue driving a car when it is unsafe to do so must be weighted against the dangers and damages the dementia-laden person can cause to other unsuspecting people and places by being behind-the-wheel. We all need to be mindful that a multi-ton vehicle can render life-or-death results when driven recklessly or irresponsibly, regardless of how sincere or well-meaning the driver might be in their heart.

Dementia Drivers And AI Autonomous Cars

What does this have to do with AI self-driving driverless autonomous cars?

At the Cybernetic AI Self-Driving Car Institute, we are developing AI software for self-driving cars. One aspect involves the AI being able to discern or attempt to discern that other driver’s on-the-road might be driving while suffering from severe dementia, and the AI should then take necessary driving precautions accordingly.

Allow me to elaborate.

I’d like to first clarify and introduce the notion that there are varying levels of AI self-driving cars. The topmost level is considered Level 5. A Level 5 self-driving car is one that is being driven by the AI and there is no human driver involved. For the design of Level 5 self-driving cars, the automakers are even removing the gas pedal, the brake pedal, and steering wheel, since those are contraptions used by human drivers. The Level 5 self-driving car is not being driven by a human and nor is there an expectation that a human driver will be present in the self-driving car. It’s all on the shoulders of the AI to drive the car.

For self-driving cars less than a Level 5 and Level 4, there must be a human driver present in the car. The human driver is currently considered the responsible party for the acts of the car. The AI and the human driver are co-sharing the driving task. In spite of this co-sharing, the human is supposed to remain fully immersed into the driving task and be ready at all times to perform the driving task. I’ve repeatedly warned about the dangers of this co-sharing arrangement and predicted it will produce many untoward results.

For my overall framework about AI self-driving cars, see my article: https://aitrends.com/selfdrivingcars/framework-ai-self-driving-driverless-cars-big-picture/

For the levels of self-driving cars, see my article: https://aitrends.com/selfdrivingcars/richter-scale-levels-self-driving-cars/

For why AI Level 5 self-driving cars are like a moonshot, see my article: https://aitrends.com/selfdrivingcars/self-driving-car-mother-ai-projects-moonshot/

For the dangers of co-sharing the driving task, see my article: https://aitrends.com/selfdrivingcars/human-back-up-drivers-for-ai-self-driving-cars/

Let’s focus herein on the true Level 5 self-driving car. Much of the comments apply to the less than Level 5 and Level 4 self-driving cars too, but the fully autonomous AI self-driving car will receive the most attention in this discussion.

Here’s the usual steps involved in the AI driving task:

  • Sensor data collection and interpretation
  • Sensor fusion
  • Virtual world model updating
  • AI action planning
  • Car controls command issuance

Another key aspect of AI self-driving cars is that they will be driving on our roadways in the midst of human driven cars too. There are some pundits of AI self-driving cars that continually refer to a Utopian world in which there are only AI self-driving cars on public roads. Currently, there are about 250+ million conventional cars in the United States alone, and those cars are not going to magically disappear or become true Level 5 AI self-driving cars overnight.

Indeed, the use of human driven cars will last for many years, likely many decades, and the advent of AI self-driving cars will occur while there are still human driven cars on the roads. This is a crucial point since this means that the AI of self-driving cars needs to be able to contend with not just other AI self-driving cars, but also contend with human driven cars. It is easy to envision a simplistic and rather unrealistic world in which all AI self-driving cars are politely interacting with each other and being civil about roadway interactions. That’s not what is going to be happening for the foreseeable future. AI self-driving cars and human driven cars will need to be able to cope with each other.

For my article about the grand convergence that has led us to this moment in time, see: https://aitrends.com/selfdrivingcars/grand-convergence-explains-rise-self-driving-cars/

See my article about the ethical dilemmas facing AI self-driving cars: https://aitrends.com/selfdrivingcars/ethically-ambiguous-self-driving-cars/

For potential regulations about AI self-driving cars, see my article: https://aitrends.com/selfdrivingcars/assessing-federal-regulations-self-driving-cars-house-bill-passed/

For my predictions about AI self-driving cars for the 2020s, 2030s, and 2040s, see my article: https://aitrends.com/selfdrivingcars/gen-z-and-the-fate-of-ai-self-driving-cars/

Returning to the topic of dementia driving, the AI of a self-driving car ought to be imbued with an ability to assess other drivers and whether they are driving in a “safe and sane” manner. Since the AI cannot somehow reach into the mind of human drivers that are on-the-road, the AI must observe the behavior of the car and infer from that observable behavior the likely state-of-mind of the human driver.

Presumably, if there’s a car up ahead that is another AI self-driving car, the AI of the AI self-driving car behind it does not need to worry as much about the AI driven car as it would of a human driven car. Some AI developers would argue that the AI of one self-driving car should actually have zero worries and zero need to observe another AI self-driving car, since the other AI self-driving car is going to always do the right thing and not make any errors that a human driver might make (so these AI developers would claim).

This perspective by some AI developers is what I refer to as an idealistic view, which I sometimes also described as an egocentric design view.

For the mind of drivers, see my article: https://aitrends.com/selfdrivingcars/motivational-ai-bounded-irrationality-self-driving-cars/

For the times when AI self-driving cars will be performing illegal driving acts, see my article: https://aitrends.com/selfdrivingcars/illegal-driving-self-driving-cars/

For when AI self-driving cars foul-up due to errors or faults, see my article: https://aitrends.com/selfdrivingcars/debugging-of-ai-self-driving-cars/

For when AI self-driving cars have the system freeze-up, see my article: https://aitrends.com/selfdrivingcars/freezing-robot-problem-and-ai-self-driving-cars/

Differing AI For Differing AI Autonomous Cars

Let’s acknowledge that once we get to true Level 5 self-driving cars, not all the respective AI’s will be the same. Different automakers and different tech firms will have developed different kinds of AI systems for their own proprietary self-driving car models. As such, each AI self-driving car model that comes from different automakers will act and react in different ways from each other.

Furthermore, since there will be different AI’s, there will be likely different ways of driving, and the AI of one self-driving car ought to be watching out for the behaviors of the AI of other self-driving cars.

That being said, I certainly concede that presumably the AI of another AI self-driving car is supposed to ultimately be more reliable, more consistent, more prone to proper driving than would be human drivers. Let me make clear that I am not suggesting that the AI only observe other AI self-driving cars, and somehow not observe human driven cars too.

I am instead clarifying and emphasizing that for those that assume the AI would only try to observe human driven cars for driving behavior, I’d argue that’s insufficient and the AI should also be observing the other AI driven cars too.

Fortunately, it will likely be easier for one AI self-driving car to directly communicate with another AI self-driving car, since they will hopefully be using in-common V2V (vehicle-to-vehicle) electronic communications. This would make things easier in the sense that the AI of one self-driving car might ask another AI as to why it just suddenly and unexpectedly changed lanes ahead, which maybe the other AI might reply that there is debris in the lane ahead and thus it then explains the seemingly odd behavior and also aids the other AI in avoiding the same debris.

Imagine if we humans were all using our cell phones while driving and continually conversing with each other. Hey you, in the red sports car ahead, why did you make that crazy right turn? Though this might be a means to aid traffic, it could also spark quite a bit of road rage. No more needing to just raise your finger to make a statement to another driver, you could speak with them directly. I’d dare say our roads would turn into boxing matches. It wouldn’t be pretty.

For my article about road rage, see: https://aitrends.com/selfdrivingcars/road-rage-and-ai-self-driving-cars/

In any case, let’s get back to the notion that the AI of your self-driving car will be observing the behavior of other cars. Doing so will aid the AI in trying to anticipate or predict what the other car might next do. By being able to make insightful predictions, the AI of your self-driving car will have a chance at being a better defensive driver and avoid untoward incidents. The AI will also be able to line-up evasive actions when needed, doing so before there is insufficient time left to react to an emerging dire situation.

What kinds of telltale clues might a dementia-laden driver provide?

Here’s some that we train our AI to be on the lookout for:

  • Riding of the brakes as exhibited by continual brake lights or slowing inexplicably
  • Pumping of the brakes repeatedly even though there is no apparent reason to do so
  • Signaling to make a right turn and then making no turn or making a left turn
  • Signaling to make a left turn and then making no turn or making a right turn
  • Turn signal continuously on for no apparent reason since no turning action is arising
  • Rolls through a stop sign
  • Speeds-up, slows down, speeds-up, slows down, but not due to traffic conditions
  • Not driving in a defensive manner and gets stuck or trapped in obvious traffic predicaments
  • Runs a red light
  • Comes to a halt in traffic when there is no apparent cause
  • Makes attempts at exits or turns and then suddenly reverts away from the attempt
  • Veers into the emergency lane or bike lane and no apparent cause to do so
  • Nearly hits other cars or pedestrians or roadway objects
  • Goes radically slower than the rest of traffic
  • Goes radically faster than the rest of traffic
  • Other cars are having to get out of the way of the observed car
  • Other cars honk their horns at the observed car or make other untoward motions
  • Keeps changing lanes when there is no apparent reason to do so
  • Cuts off other cars when changing lanes and making other maneuvers
  • Other

Caveats About Dementia Driving Behaviors

Please make sure to review this dementia-laden driving symptoms list with a grain of salt.

I am sure all of us have performed one or more of those kinds of driving actions from time-to-time. Maybe you are groggy from that late-night partying and in the morning your driving is not at your usual peak performance. Maybe you are in a foul mood and taking it out on the rest of the traffic. Plus, novice teenage driver often performs those same moves, primarily because they are still wrestling with the basics of driving and aren’t sure of what they are doing.

The notion is that any of those driving actions in isolation could be due to any number of reasons. I once had a bee that got into my car while I was driving, and I regrettably weaved across the lanes as I was trying to get the scary critter out of my car. A momentary act that appears out of the ordinary should be construed as a potential warning that perhaps the driver is somehow amiss, but it usually takes more than one single act to fully make it onto the “watch out for that car” mindset (unless the single act is so egregious that it is clear cut that something bad is happening).

You might be wondering what the big deal is about detecting a car that has these kinds of foul driving actions?

The odds are that once you spot this kind of behavior emerging, it will likely continue if the driver has some systemic issues involved in their driving. This gives the AI a heads-up to be especially wary of that car.

For example, if the AI detected that a car ahead was needlessly riding its brakes, this might be a sign that the driver might soon take some other dangerous action such as a wild turn or veering into other lanes. The AI would then anticipate this possibility and potentially change the path of the self-driving car. It might be safer to route to another road or perhaps let the car ahead get some distance between the AI self-driving car and it. These are all prudent defensive driving actions by the AI and would be spurred when a car appears to be driven in an untoward manner.

Some of you might be saying that these kinds of driving moves could be undertaken by a drunk driver. You are indeed right! I would suggest that a drunk driver could do any or all of those kinds of driving moves. A drunk driver might do those and even go further and make even worse moves. Can you for sure distinguish between a drunk driver and a dementia-laden driver, based on the behavior exhibited by the car’s actions? It is hard to assert that you could make such a distinction without otherwise scrutinizing the actual human driver to figure out what is afoot.

For more about drunk driving, see my article: https://aitrends.com/selfdrivingcars/dui-drunk-driving-self-driving-cars-prevention-cure/

For my article about Machine Learning and AI self-driving cars, see: https://aitrends.com/ai-insider/ensemble-machine-learning-for-ai-self-driving-cars/

For safety issues of AI self-driving cars, see my article: https://aitrends.com/ai-insider/safety-and-ai-self-driving-cars-world-safety-summit-on-autonomous-tech/

For how AI self-driving cars can sometimes drive erratically too, see my article: https://aitrends.com/ai-insider/ghosts-in-ai-self-driving-cars/

For swarms of AI self-driving cars, see my article: https://aitrends.com/selfdrivingcars/swarm-intelligence-ai-self-driving-cars-stigmergy-boids/

If an AI self-driving car is able to detect a potential dementia-laden driver, it could try to alert other nearby AI self-driving cars about the matter. Using the V2V, the AI might send a message to be on-the-watch for a blue sedan that is at the corner of Main and Sprout street and heading west. Other AI self-driving cars would then be able to likewise be prepared for evasive action. There is even the possibility of using a swarm-like approach to provide a safety driving traffic cocoon for the driver.

I realize that this seems a bit like Big Brother to have other cars watching for and then taking semi-collective action about another driver that is on-the-road. I would claim though that this already happens to some extent with human drivers acting at times in a collective manner.

In the case of the grandfather, the other drivers in the neighborhood knew that he was a driver that was increasingly getting worse and worse. They would often “shield” his driving by purposely driving near to him and helping to clear traffic nearby. It was almost like a parade of cars, but the “star” of the parade was not even aware that his fellow neighbors were taking such an action (which reinforces that his dementia was bad enough that he couldn’t discern what the other traffic was doing for him).

Some drivers that have dementia will at least try to minimize their chances of getting themselves into trouble. For example, if they have an especially difficult time when driving in a location they do not know, they will drive only on streets they do know. If they have a difficult time comprehending traffic at nighttime, they will purposely only drive during daylight. If they know that lots of other traffic confounds them, they’ll wait until the least traffic periods to then get onto the roadway. Etc.

Ultimately, if the dementia overtakes the ability to appropriately drive a car, something will need to be done to ensure that the person does not get behind the wheel. The so-called “taking away the keys” has got to be one of the hardest acts to undertake. It is hard for the person that is forfeiting their keys and the privilege of driving. It is hard for whomever has to take away the keys. The matter can create ill will and taint the rest of the person’s existence.

The good news is that with the advent of true Level 5 self-driving cars, it is anticipated that those with dementia will still be able to have the mobility they crave, simply by using AI self-driving cars to get them where they want to go. Sure, they won’t be able to get behind the wheel of the car, but I think they’ll accept the notion of being a passenger rather than a driver, particularly due to the aspect that lots and lots of other people will be doing so too. In other words, those people getting into AI self-driving cars will include many that could drive if they wished, and instead they prefer to let the driving be done by the AI self-driving car. The person with dementia won’t stand out as someone using an AI self-driving car since we’ll all be routinely using AI self-driving cars.

For more about ridesharing and AI self-driving cars, see my article: https://aitrends.com/selfdrivingcars/ridesharing-services-and-ai-self-driving-cars-notably-uber-in-or-uber-out/

For my article about the privacy aspects of AI self-driving cars, see: https://aitrends.com/selfdrivingcars/privacy-ai-self-driving-cars/

For future jobs involving aiding others that are using AI self-driving cars, see my article: https://aitrends.com/selfdrivingcars/future-jobs-and-ai-self-driving-cars/

For the possibility of becoming addicted to using AI self-driving cars, see my article: https://aitrends.com/selfdrivingcars/addicted-to-ai-self-driving-cars/

Conclusion

Family members and friends are usually the first to realize that someone is succumbing to dementia. Allowing an untoward driver onto the roadways is nearly the same as letting a drunk driver onto the road. Most of us would likely try to stop someone that is drunk from getting behind the wheel. It’s easier, of course to do so since it is likely a one-time stopping action and not something of a more permanent nature.

The person with dementia will eventually reach a crossover point that makes their driving dangerous for themselves and dangerous for everyone else. Hopefully, if you do need to intervene and take away the keys, the advent of AI self-driving cars will have become so pervasive that their shifting into a ride sharing mode of using AI self-driving cars will ease the agony of losing the privilege to drive a car.

Since we will have a mixture of both human driven cars and AI self-driving cars for a long time, you’ll unfortunately still need to be ready to be the gatekeeper of dealing with the key’s removal aspects. In any case, the AI of the self-driving car has to be savvy enough to be watchful for dementia-laden drivers and take the kinds of evasive actions to save the lives of those intertwined in traffic with that untoward driver. I think we can all agree we’d want the AI to be watchful and have the capability to contend with these potentially life-and-death matters.

Copyright 2020 Dr. Lance Eliot

This content is originally posted on AI Trends.

[Ed. Note: For reader’s interested in Dr. Eliot’s ongoing business analyses about the advent of self-driving cars, see his online Forbes column: https://forbes.com/sites/lanceeliot/]

AI Driving Personalization Efforts at Restaurants; Dynamic Drive-Thru Menus Coming

By AI Trends Staff

AI is no longer an under-the-radar experiment in the restaurant business, with many chains making substantial investments and headway with AI apps that seek to advance customer personalization.

Taco Bell, for example, is incorporating AI into its mobile app used by five million customers, through a partnership with Certona, a personalization engine, according to a recent account in Forbes. The app relies on machine learning to present content to users based on their individual behavior. It can differentiate menu items and pricing based on geographic region.

“Instead of displaying generic, or static, product recommendations, we use Certona’s AI engine to determine what the best products are to display to a customer,” stated Derrick Chan, Taco Bell’s director of e-commerce. “So, for example, if it is a first-time user, we are mostly factoring in overall sales data, such as what four items best sell with the item currently being viewed.” Chan said.

He compared it to how Amazon provides product recommendations. “We’re just doing the same thing for food,” Chan stated.

Derrick Chan, Director of E-Commerce, Taco Bell

The technology will be tested when its capabilities are available to mobile-heavy customers. Chan is optimistic about its potential. “With that personalization and relevancy comes exposure to new menu offerings, customizations and opportunities for our fans to engage with the brand in a new and innovative way,” he stated.

Many Restaurants Moving to Adopt AI to Gain an Advantage

Other restaurants exploring the use of AI to better their business include: Chick-fil-A, which is using technology to help identify food safety issues; TGI Fridays, using it to try to increase volume on popular offerings, including alcohol; and McDonald’s, which acquired Dynamic Yield last year for $300 million.

The Israeli startup used personalization and decision logic to create a more personalized experience that could include new outdoor digital drive-thru menu displays that can change based on time, weather and trending menu items. McDonald’s tested the technology in several US restaurants in 2018; it plans to roll it out across the US and then expand into international markets. It plans to integrate the Dynamic Yield technology into all its digital customer experience touch points, including a mobile app.

Soon after, McDonald’s announced the acquisition of Appente, which uses AI to help understand voice commands.

Elsewhere, Starbucks President and CEO Kevin Johnson recently called the company’s Deep Brew (AI) technology a “key part” of Starbucks’ future. The technology calculates a store’s inventory requirements and predicts how many baristas are needed behind the counter every day, according to an account in Restaurant Dive.

Also, Domino’s has reported on its AI efforts using Nvidia GPUs. The company uses the system to predict when an order will be ready, improving the accuracy rate to 95%.

Good Times Burger & Frozen Custard of Colorado has implemented Valyant AI at its drive-throughs. Valyant CEO Rob Carpenter told Restaurant Dive that an AI product can increase profitability 10% to 30% by increasing throughput, upselling, automating mundane tasks, and providing other labor savings.

Starbucks Partnering with Microsoft

Starbucks is partnering with Microsoft to use AI to enhance customer service and drive sales.

Starbucks chief technology officer Gerri Martin-Flickinger stated in an account in The Motley Fool, “As an engineering and technology organization, one of the areas we are incredibly excited to be pursuing is using data to continuously improve the experience for our customers and partners.”

Gerri Martin-Flickinger, CTO, Starbucks

When customers use the mobile app, the company collects data about preferences. Combined with a knowledge of Starbucks shops in an area, local popular drinks, the weather and other factors, the app can offer recommendations for products and pairings.

Starbucks also offers a mobile order-and-pay app, with which customers can order on their phones, prepay, and then pick up their orders from their local Starbucks shop. This is now the second most popular mobile-payment app behind Apple Pay based on user totals, reported the Motley Fool.

Starbucks has more than 14,000 US locations and approximately 30,000 stories worldwide. It’s most recent quarterly revenue was reported at $4.6 billion.

Starbucks Senior Vice President Jon Francis said, “We’re meeting our customers where they are… using machine learning and artificial intelligence to understand and anticipate their personal preferences. Machine learning also plays a role in how we think about store design, engage with our partners, optimize inventory and create barista schedules. This capability will eventually touch all facets of how we run our business.”

Read the source articles in Forbes,  Restaurant Dive and The Motley Fool.

As in Business, Cloud Strategy for Government Agencies Must Be a Fit

By AI Trends Staff

Public sector technology executives know they need to modernize, while accounting for cybersecurity and trying to reduce costs. This often results in a push to pursue a cloud strategy, and in doing so, the same principles apply for a federal government agency as for a private business: find the best fit.

The General Services Administration (GSA) is working through a cloud transformation. GSA CIO Dave Shive, in an account from Meritalk, said the cloud strategy needs to fit the business needs and take agency resources into account. He can provide lessons in setting up a transition based on his agency’s experience.

“There’s no one size fits all. Cloud does and will continue to look different for every agency,” Shive stated at Cloudera’s recent Data Cloud Summit. “Agencies should take time to resource, scale, cloud adoption—both time and resources—to best meet their respective missions.”

David Shive, CIO, GSA

The GSA ended up closing all 121 of its data centers to improve on its infrastructure strategy.

The workforce was reallocated to focus on higher value outcomes. If the target strategy is to adopt cloud,  “Just get started,” Shive recommended.

He made the following suggestions for making a cloud move successful:

  • Increase talent level through training;
  • Have strong leadership support;
  • Determine what kind of data there is and the sensitivity of the data that will move to cloud;
  • Be smart when acquiring cloud technology because “not all cloud is created equal;” and
  • Ensure that stakeholders understand the values of IT investments.

Emphasize Outcomes Over Technology

Be careful not to emphasize technology over outcomes, advised Mark Forman, Vice President, Digital Government for Unisys Federal, in a recent account in Nextgov. Getting desired results from the tidal shift to cloud computing will require a new standard of management. Moving applications to the cloud will not guarantee benefits.

Mark Forman, Vice President, Digital Government for Unisys Federal

Savvy IT shops are looking toward an “ecosystem operating model focused on outcomes, as opposed to the “IT silo” approach of managing unique software and hardware for each application. The new operating model encompasses:

  • Secure hybrid cloud;
  • Cloud management platform that enables the CIO as a trusted broker;
  • Transparency of IT service costs and performance;
  • Deploying web-services and micro-services to replace inflexible applications components;
  • IT process automation; and
  • DevSecOps for process digitization instead of automating “cow paths.”

CIOs should move away from the capital expenditure model and toward an IT service catalog. Document performance, security and cost in order to compare to legacy approaches. A cloud management platform should be used to compare options, for ordering, and tracking data on usage, performance, and costs.

Employ agile development infused with security, and engage line workers. This is to align with DevOps, a set of practices that combine software development and IT operations, with the aim of shortening the systems development life cycle and provide continuous, high-quality delivery.

Be prepared for systems owners who need to overcome fear about loss of control. Having an outcome-based system for showing cost and performance advantages is a good idea. Given the shifting dynamics of cloud offerings, CIOs must continually adjust their IT service catalogs to encourage adoption while accepting a certain level of fixed costs.

Research and advisory firm Gartner recommends organizations keep working on their cloud-first strategy.  “If you have not developed a cloud-first strategy yet, you are likely falling behind your competitors,” stated Elias Khnaser, VP Analyst at Gartner, in a recent article on planning a cloud strategy. “IT organizations have moved past asking whether applications can be deployed or migrated to the public cloud. Instead, they are commonly accepting the pace and innovation of cloud providers as foundational to their business.”

This means the cloud-first strategy needs to be embraced by the whole organization, not just IT. For some organizations, moving all applications out of the data centers might be the way to go. For others, moving a subset of applications to the public cloud might be the best approach.

Practice workload placement analysis, involving reassessing workloads on a regular basis, always assessing whether to stick with the current execution venue or move to an alternative with higher value without adding significant risk.

A multi-cloud strategy offers more options and is more challenging to manage. Organizations need visibility into the cost of computer services being consumed.  They must govern consumption of cloud services by provider, and consumption across cloud providers in order to effectively manage the environment.

This requires a cloud management tooling strategy to minimize the number of tools needed in order to fulfill the management objectives. “The best strategy is a combination of solutions, based on the required degrees of cross-platform consistency and platform-specific functionality,” stated Khnaser. “In all cases, organizations should prioritize the use of the cloud platform’s native toolset, augmenting that where needed with third-party cloud management platforms, cloud management point tools, DIY solutions and outsourcing.”

Read the source articles in Meritalk, Nextgov and at Gartner.

Thursday, 30 January 2020

What Is a Data Lakehouse?

Over the past few years at Databricks, we’ve seen a new data management paradigm that emerged independently across many customers and use cases: the data lakehouse. In this post we describe this new system and its advantages over previous technologies.

Data warehouses have a long history in decision support and business intelligence applications. Since its inception in the late 1980s, data warehouse technology continued to evolve and MPP architectures led to systems that were able to handle larger data sizes. But while warehouses were great for structured data, a lot of modern enterprises have to deal with unstructured, semi structured, and data with high variety, velocity, and volume. Data warehouses are not suited for many of these use cases, and they are certainly not the most cost efficient.

As companies began to collect large amounts of data from many different sources, architects began envisioning a single system to house data for many different analytic products and workloads. About a decade ago companies began building data lakes – repositories for raw data in a variety of formats. While suitable for storing data, data lakes lack some critical features: they do not support transactions, they do not enforce data quality, and their lack of consistency / isolation makes it almost impossible to mix appends and reads, and batch and streaming jobs.

 

The need for a flexible, high-performance system hasn’t abated. Companies require systems for diverse data applications including BI and analytics, real-time monitoring, data science, and machine learning. Most of the recent advances in AI have been in better models to process unstructured data (text, images, video, audio), but these are precisely the types of data that a data warehouse is not optimized for. A common approach is to use multiple systems – a data lake, several data warehouses, and other specialized systems such as streaming, time-series, graph, and image databases. Having a multitude of systems introduces complexity and more importantly, introduces delay as data professionals invariably need to move or copy data between different systems.

Evolution of data storage, from data warehouses to data lakes to data lakehouses

What is a data lakehouse?

New systems are beginning to emerge that address the limitations of data lakes. A data lakehouse is a new paradigm that combines the best elements of data lakes and data warehouses. Data lakehouses are enabled by a new system design: implementing similar data structures and data management features to those in a data warehouse, directly on the kind of low cost storage used for data lakes. They are what you would get if you had to redesign data warehouses in the modern world, now that cheap and highly reliable storage (in the form of object stores) are available.

A data lakehouse has the following key features:

  • Storage is decoupled from compute: In practice this means storage and compute use separate clusters, thus these systems are able to scale to many more concurrent users and larger data sizes. Some modern data warehouses also have this property.
  • Openness: The storage formats they use are open and they provide an API so different tools and engines can access data. For example, existing data lakehouses enable using BI tools directly on the source data. This reduces staleness and improves recency, reduces latency, and lowers the cost of having to operationalize two copies of the data in both a data lake and a warehouse.
  • Support for diverse data types ranging from unstructured to structured data: The data lakehouse supports SQL and can house relational data including star-schemas commonly used in data warehouses. In addition they can be used to store, refine, analyze, and access data types needed for many new data applications, including images, video, audio, semi-structured data, and text.
  • Support for diverse workloads: including SQL and analytics, data science, and machine learning. Multiple tools might be needed to support all these workloads but they all rely on the same data repository.
  • Transaction support: In an enterprise data lakehouse many data pipelines will often be reading and writing data concurrently. Support for ACID transactions ensures that as multiple parties concurrently read or write data, the system is able to reason about data integrity.
  • End-to-end streaming: Real-time reports are the norm in many enterprises. Support for streaming eliminates the need for separate systems dedicated to serving real-time data applications.

These are the key attributes of data lakehouses. Enterprise grade systems require additional features. Tools for security and access control are basic requirements. Data governance capabilities including auditing, retention, and lineage have become essential particularly in light of recent privacy regulations. Tools that enable data discovery such as data catalogs and data usage metrics are also needed. With a data lakehouse, such enterprise features only need to be implemented, tested, and administered for a single system.

Some early examples

The Databricks Platform has the architectural features of a data lakehouse. Microsoft’s Azure Synapse Analytics service, which integrates with Azure Databricks, enables a similar lakehouse pattern. Other managed services such as BigQuery and Redshift Spectrum have some of the lakehouse features listed above, but they are examples that focus primarily on BI and other SQL applications. Companies who want to build and implement their own systems have access to open source file formats (Delta Lake, Apache Iceberg, Apache Hudi) that are suitable for building a data lakehouse.

Merging data lakes and data warehouses into a single system means that data teams can move faster as they are able use data without needing to access multiple systems. The level of SQL support and integration with BI tools among these early data lakehouses are generally sufficient for most enterprise data warehouses. Materialized views and stored procedures are available but users may need to employ other mechanisms that aren’t equivalent to those found in traditional data warehouses. The latter is particularly important for “lift and shift scenarios”, which require systems that achieve semantics that are almost identical to those of older, commercial data warehouses.

What about support for other types of data applications? Users of a data lakehouse have access to a variety of standard tools (Spark, Python, R, machine learning libraries) for non BI workloads like data science and machine learning. Data exploration and refinement are standard for many analytic and data science applications. Delta Lake is designed to let users incrementally improve the quality of data in their lakehouse until it is ready for consumption.

A note about technical building blocks. While distributed file systems can be used for the storage layer, objects stores are more commonly used in data lakehouses. Object stores provide low cost, highly available storage, that excel at massively parallel reads – an essential requirement for modern data warehouses.

From BI to AI

The data lakehouse is a new data management paradigm that radically simplifies enterprise data infrastructure and accelerates innovation in an age when machine learning is poised to disrupt every industry. In the past most of the data that went into a company’s products or decision making was structured data from operational systems, whereas today, many products incorporate AI in the form of computer vision and speech models, text mining, and others. Why use a data lakehouse instead of a data lake for AI? A data lakehouse gives you data versioning, governance, security and ACID properties that are needed even for unstructured data.

Current data lakehouses reduce cost but their performance can still lag specialized systems (such as data warehouses) that have years of investments and real-world deployments behind them. Users may favor certain tools (BI tools, IDEs, notebooks) over others so data lakehouses will also need to improve their UX and their connectors to popular tools so they can appeal to a variety of personas. These and other issues will be addressed as the technology continues to mature and develop. Over time data lakehouses will close these gaps while retaining the core properties of being simpler, more cost efficient, and more capable of serving diverse data applications.

--

Try Databricks for free. Get started today.

The post What Is a Data Lakehouse? appeared first on Databricks.