Friday, 28 February 2020

Securely Accessing Azure Data Sources from Azure Databricks

Azure Databricks is a Unified Data Analytics Platform that is a part of the Microsoft Azure Cloud. Built upon the foundations of Delta Lake, MLFlow , Koalas and Apache Spark, Azure Databricks is a first party service on Microsoft Azure cloud that provides one-click setup, native integrations with other Azure services, interactive workspace, and enterprise-grade security to power Data & AI use cases for small to large global customers. The platform enables true collaboration between different data personas in any enterprise, like Data Engineers, Data Scientists, Data Analysts and SecOps / Cloud Engineering.

In this blog which is first in a series of two, we’ll provide an overview of Azure Databricks architecture and how customers could connect to their own-managed instances of Azure data services in a secure manner.

Azure Databricks Architecture Overview

Azure Databricks is a managed application on Azure cloud. At a high-level, the architecture consists of a control / management plane and data plane. The control plane resides in a Microsoft-managed subscription and houses services such as web application, cluster manager, jobs service etc. In the default deployment, the data plane is a fully managed component in customer’s subscription that includes a VNET, NSG and a root storage account known as DBFS.

The data plane could also be deployed in a customer-managed VNET, to allow the SecOps and Cloud Engineering teams build security & network architecture for the service as per their enterprise governance policies. This capability is called Bring Your Own VNET or VNET Injection. The picture shows a representative view of such customer architecture.

Azure Databricks is a managed application on Azure cloud. At a high-level, the architecture consists of a control / management plane and data plane.

Secure connectivity to Azure Data Services

Enterprise Security is a core tenet of building software at both Databricks and Microsoft, and thus it’s considered as a first-class citizen in Azure Databricks. In the context of this blog, secure connectivity refers to ensuring that traffic from Azure Databricks to Azure data services remains on the Azure network backbone, with the inherent ability to whitelist Azure Databricks as an allowed source. As a security best practice, we recommend a couple of options which customers could use to establish such a data access mechanism to Azure Data services like Azure Blob Storage, Azure Data Lake Store Gen2, Azure Synapse Data Warehouse, Azure CosmosDB etc. Please read further for a discussion on Azure Private Link and Service Endpoints.

Option 1: Azure Private link

The most secure way to access Azure Data services from Azure Databricks is by configuring Private Link. As per Azure documentation – Private Link enables you to access Azure PaaS Services (for example, Azure Storage, Azure Cosmos DB, and SQL Database) and Azure hosted customer/partner services over a Private Endpoint in your virtual network. Traffic between your virtual network and the service traverses over the Microsoft network backbone, eliminating exposure from the public Internet. You can also create your own Private Link Service in your virtual network (VNet) and deliver it privately to your customers. The setup and consumption experience using Azure Private Link is consistent across Azure PaaS, customer-owned, and shared partner services. For details, please refer to this.

See below on how Azure Databricks and Private Link could be used together.

Azure Databricks and Azure Data Service Private Endpoints in separate VNETs

Azure Databricks and Azure Data Service Private Endpoints in separate VNETs

ALT TAG = Azure Databricks and Azure Data Service Private Endpoints in same VNET

Azure Databricks and Azure Data Service Private Endpoints in same VNET

Private Endpoint Considerations

Please consider the following before implementing the private endpoint:

  • Provides protection against data exfiltration by default. In the case of Azure Databricks, this would apply once customer whitelists access to specific services in the control plane.
  • Keeps traffic on Azure network backbone i.e public network is not used for any data flow.
  • Extends your private network address space to Azure Data services, i.e. the Azure data service effectively gets a private IP in one of your VNETs and could be treated as part of your larger private network.
  • Connect privately to Azure Data services in other regions i.e. VNET in region A could connect to endpoints in region B via Private Link.
  • Private Link is relatively bit more complex to set up as compared to other secure access mechanisms.
  • See the documentation for a detailed list of Private Link benefits and the service specific availability.

One example of where one could use Private Link is when a customer uses a few Azure Data services in production along with Azure Databricks, like Blob Storage, ADLS Gen2, SQL DB etc. The business would like the users to query the masked aggregated data from ADLS Gen2, but restrict them from making their way to the unmasked confidential data in other data sources. In that case, a private endpoint could be established only for ADLS Gen2 service using any of the sub-options discussed above.

This is how such an environment could be configured:

1 – Setup Private Link for ADLS Gen2

2 – Deploy Azure Databricks in your VNET

Please note that it’s possible to configure more than one Private Link per Azure Data service, which allows you to build an architecture that conforms to your enterprise governance needs.

Option 2: Azure Virtual Network Service Endpoints

As per Azure documentation, Virtual Network (VNET) service endpoints extend your virtual network private address space. The endpoints also extend the identity of your VNet to the Azure services over a direct connection. Endpoints allow you to secure your critical Azure service resources to only your virtual networks. Traffic from your VNet to the Azure service always remains on the Microsoft Azure network backbone.

Service endpoints provide the following benefits (source):

Improved security for your Azure service resources

Private address space for different virtual networks can overlap with each other. You can’t use overlapping network space to uniquely identify traffic that originates from a particular VNET. Once service endpoints are enabled for the subnets in your VNET, you can add a virtual network firewall rule to secure the Azure data services by extending your VNET identity to those resources. Such a configuration helps remove public access to those resources and allowing traffic only from your VNET.

Optimal routing for Azure data service traffic from your virtual network

Today, any routes on your VNET that are used to direct public network-headed traffic via your cloud/on-premises-based virtual appliances are also used for the Azure data service traffic. Service endpoints provide optimal routing for Azure traffic.

Keeping traffic on the Azure network backbone

Service endpoints always direct Azure data service traffic directly from your VNET to the resource on the Microsoft Azure network backbone. Keeping traffic on the Azure network backbone allows you to continue auditing and monitoring outbound Internet traffic from your virtual networks, through forced-tunneling, without impacting data service traffic. For more information about user-defined routes and forced-tunneling, see Azure virtual network traffic routing.

Simple to set up with no management overhead

You no longer need reserved, public IP addresses in your virtual networks to secure Azure data service resources through IP firewall. There are no Network Address Translation (NAT) or gateway devices required to set up the service endpoints. You can configure service endpoints through a simple setup for a subnet. There’s no additional overhead to maintaining the endpoints.

Azure Service Endpoint with Azure Databricks

Azure Service Endpoint with Azure Databricks

Azure Service Endpoint Considerations

Please consider the following before implementing the service endpoints:

  • Does not provide protection against data exfiltration by default.
  • Keeps traffic on Azure network backbone i.e public network is not used for any data flow.
  • Does not extend your private network address space to Azure Data services.
  • Cannot connect privately to Azure Data services in other regions (except for paired regions).
  • See the documentation for a detailed list of Azure Service Endpoint benefits and limitations.

Taking the same example as mentioned above for Private Link, and how it could look like with Service Endpoints. In this case, Azure Storage Service Endpoint could be configured on Azure Databricks subnets and the same subnets could then be whitelisted in ADLS Gen2 firewall rules.

This is how such an environment could be configured:

1 – Setup Service Endpoint for ADLS Gen2

2 – Deploy Azure Databricks in your VNET

3 – Configure IP firewall rules on ADLS Gen2

Getting Started with Secure Azure Data Access

We discussed a couple of options available to access Azure data services securely from your Azure Databricks environment. Based on your business specifics, you could either use Azure Private Link or Virtual Network Service Endpoints. Once the network connectivity approach is finalized, you could utilize secure auth approaches to connect to those resources:

In the next blog in this series, we’ll dive deep into how one could set up a buttoned-up locked down environment to prevent data exfiltration (in other words, implement a data loss prevention architecture). It would utilize a mix of the above discussed options and Azure Firewall. Please reach out to your Microsoft or Databricks account teams for any questions.

--

Try Databricks for free. Get started today.

The post Securely Accessing Azure Data Sources from Azure Databricks appeared first on Databricks.

Check out the killer lineup of keynotes at Spark + AI Summit 2020

The Spark + AI Summit is already the world’s largest data and machine learning conference bringing together engineers, scientists, developers, analysts and leaders from around the world.

This year is shaping up to be our biggest conference ever, with over 7,000 attendees expected to attend four days of training sessions, presentations and networking events. We’ve also expanded our keynote lineup this year to include data and machine learning innovators and visionaries from the media, academia and open source.

Who’s Keynoting?

Spark + AI 2020 Featured Keynote Speakers Nate Silver, Jennifer Chayes, and Adam Paske

Databricks executives and original creators of popular open source projects including Apache Spark, Delta Lake, MLflow, and Koalas will also hit the keynote stage:

Hundreds of other Data and ML Sessions, Tutorials and Training Classes

This year, we’re excited to have Ben Lorica join as the Program Chair. Ben is the former Chief Data Scientist at O’Reilly Media, and the former Program Chair of: the Strata Data Conference, the O’Reilly Artificial Intelligence Conference, and TensorFlow World.

Now with Summit expanded to four days, Ben and the rest of the program team are combing through the vast array of community submissions to build a compelling agenda. Stay tuned to the Databricks Blog for the complete schedule to be announced soon.

Spark + AI 2020 Best in Class training

Join Us in San Francisco for Spark + AI Summit 2020

We hope you’ll join us at the Spark + AI Summit 2020! Register now to save an extra 20% off the already low early-bird rate — use code RBlogSAI2

--

Try Databricks for free. Get started today.

The post Check out the killer lineup of keynotes at Spark + AI Summit 2020 appeared first on Databricks.

Top Five HR Trends for 2020

As we wrap up this year and going ahead in a new decade, the future of work is purpose-driven and people-focused. Over the course of the 2010s, the space of human resources has extended and progressed alongside changes in how organizations function, how teams are managed, and how employees set their expectations when it comes to working.

As we enter the next decade, HR must put their prime focus on elements like- people management and human resources to providing- engaging and exceptional work experiences for their people.

Here are five HR trends for 2020 that will transform the workforce and the workplace in 2020.

Trend #1 Using People analytics for decision-making

We all know that big data analysis has a wide-reaching potential that can quickly drive any branch of an organization. In HR, applying a data-driven approach to people analytics enable employers to come up with valuable insights on- employee performance and satisfaction and that influences their motivation and productivity.

In the current scenario, HR departments are equipped with information that you can avail easily, like- employee stats, recruitment data, performance KPIs, etc. In the coming years, organizations will apply this valuable technology for smart decisions for their companies and their current employees.

Trend #2 Employer ...


Read More on Datafloq

How To Own Your Next Project

Nowadays, Agile has a tremendous potential advantage over businesses, it's important not to get lost in some of the hype that exists about Agile management style. One must also need to take more time to understand what problem Agile can potentially solve and how it will benefit your business.Agile can pose challenges for many companies and project managers. Many organizations have an existing management system or a project management approach that is based on a traditional, plan-driven method or what is sometimes called the “Waterfall model”. Once the Project managers are in, they revolve around different choices like whether to continue with the traditional approach which has an emphasis on managing costs and schedules or move with the Agile management approach which can offer many benefits including faster time-to-market and higher business value. However, both have their risks associated with either losing control of projects or less focus on managing costs and schedules.The major problem is that many companies see the agile management approach as a better option, while they make mistakes of attempting to force-fit their projects and business to either one of these extremes. It’s like attempting to plug an appliance requiring AC power into a DC outlet. ...


Read More on Datafloq

Chatbot vs. Intelligent Virtual Assistant: 9 Ways to Tell the Difference

Most businesses today are trying to implement a conversational solution to help improve overall Customer Experience. And if you are a Customer Experience or Digital Transformation professional looking for one, you might be a little confused about the difference between ‘chatbots’ and ‘virtual assistants’, among a host of other terms used for such solutions. 



Intelligent Virtual Assistant (IVA) is a term that has entered common usage over the past year as a means of discerning ‘good chatbots’ vs’ bad chatbots’. Broadly speaking, the term ‘chatbot’ is typically used for a solution that can handle only simple, routine queries and FAQs. ‘Intelligent Virtual Assistants’, on the other hand, are more advanced conversational solutions – equipped with NLU (Natural Language Understanding), NLG (Natural Language Generation), and Deep Learning, that enables them to understand and retain context and have more productive conversations with users. 



While the terms ‘chatbot’ and ‘Intelligent Virtual Assistant are still sometimes used interchangeably by customers and businesses alike, there are significant differences between the two in terms of scope, complexity and capability, all of which are far greater in IVAs as compared to chatbots. Chatbots can simulate a conversational experience to a certain extent, but are ultimately constrained by having to ...


Read More on Datafloq

HPCL sets up first electric vehicle charging station in Gujarat's Vadodara

With the installation of public charging stations, the range anxiety of EV owners is expected to reduce, which will increase the adoption of electric mobility.

AI Being Applied to Optimize Electric Battery Recharging

By AI Trends Staff

A team of researchers from Stanford University, MIT and the Toyota Research Institute have used AI to dramatically speed up the time required to test and optimally charge batteries for electric vehicles (EVs).

As recently reported in Nature, Stanford professors Stefano Ermon and William Chueh sought ways to charge an EV battery more quickly while maximizing the overall battery life. The study showed how a patented AI program could predict different ways batteries would react to charging methods.

The software also decided in real time what charging approaches to focus on or ignore. The researchers cut the testing process from two years to 16 days by reducing the length and number of trials.

Stefano Ermon, Professor of Computer Science, Stanford University

The machine learning system was trained on data of batteries that failed. It was able to detect patterns for predicting how long batteries would last.

This resulted in a new fast-charging protocol, which showed how to optimize battery life. Using AI in battery testing is a new approach, according to the researchers.

“When talking to material scientists and people who work in batteries for a living, we realized that nobody was actually using more sophisticated AI in this space, so we thought it was promising,” stated Ermon, a professor of computer science at Stanford, in an interview published in TechRepublic.

He described the many ways to charge a battery. “You can apply different voltages, different currents, different intensities––they may all charge the battery in the same amount of time, but some might harm the internal components of the battery,” he stated. “Depending on what kind of charging protocol you use, that can significantly affect the life of the battery.”

Major EV manufacturers may take an interest, Ermon predicted.

“We figured out how to greatly accelerate the testing process for extreme fast charging,” stated Peter Attia, who participated in the study as a graduate student, in an interview with SciTechDaily. “What’s really exciting, though, is the method. We can apply this approach to many other problems that, right now, are holding back battery development for months or years.”

“Machine learning is trial-and-error, but in a smarter way,” stated Aditya Grover, a graduate student in computer science who also participated in the study. “Computers are far better than us at figuring out when to explore – try new and different approaches – and when to exploit, or zero in, on the most promising ones.”

Ermon stated, “It gave us this surprisingly simple charging protocol – something we didn’t expect. That’s the difference between a human and a machine: The machine is not biased by human intuition, which is powerful but sometimes misleading.”

Wider Application Seen

The approach has the potential to accelerate every piece of the battery development pipeline, from designing the chemistry of a batter, to determining its size and shape, to finding better systems for manufacturing and storing, the researchers suggested. This has implications not only for EV battery charging but for other types of energy storage, such as for wind and solar power.

“This is a new way of doing battery development,” stated Patrick Herring, a co-author of the study and a scientist at the Toyota Research Institute. “Having data that you can share among a large number of people in academia and industry, and that is automatically analyzed, enables much faster innovation.”

The researchers intend to make the study’s machine learning and data collection system available for future battery scientists to freely use.

Ermon suggested other big data testing problems, from drug development to optimizing the performance of X-rays and lasers, could be revolutionized by the use of machine learning optimization.

Private industry has been working on applying AI to battery charging as well. Researchers at battery company StoreDot have been using machine learning to extend its capabilities, wrote Dr. Doron Myersford, CEO of StoreDot, in a recent account in Engineering and Technology.

“An initial foray into this technique has achieved remarkable results,” he stated, resulting in a decision to dedicate an R&D team to building capabilities in machine learning. The plan is to apply the lessons learned to the company’s next generation of EV batteries. He cautioned, “Ultra-fast charging presents a very complex issue,” involving innovative data science combined with expertise in electrochemistry, cell structure, anodes, cathodes and electrolytes, so more complex conclusions can be reached.

In other battery research efforts, the search is on for new materials that can store more energy than the graphite anode in modern lithium-ion batteries, according to a recent account in Battery Power Online. Rechargeable batteries with lithium metal anodes could represent the ultimate limit in energy density; however, they face major technical and safety hurdles. The high energy density means they are prone to react with other components in a battery cell to break down through large volume changes. They also run the risk of short circuiting, causing rapid heat generation and potential fire or explosion. Research is continuing.

Read the source articles in Nature, TechRepublic, SciTechDaily, Engineering and Technology and Battery Power Online.

AI Put to Work to Help US Steel Industry Stay Competitive

By AI Trends Staff

As the US steel industry looks for ways to lower costs in a global market facing slowing demand, a modern steel plant in Arkansas is using AI to help it become more competitive.

The Big River Steel Mill, which began operating in January 2017, melts scrap metal and produces steel for more than 200 customers, including four automakers, according to a recent account in  WSJPro.

The plant’s AI system has been designed by Noodle Analytics of San Francisco, which uses deep learning and neural networks to continually train algorithms on data captured by thousands of sensors.

“We’re using the best available technology and pressing that technology farther, we think, than anyone in the steel industry,” stated Big River Chief Executive David Stickler, a veteran of the steel, mining and recycling industries. “Any future steel facilities that are built will try to capitalize on what we’ve done and replicate it.”

An environment of falling steel prices and a decline in demand from manufacturers is creating  an opportunity for newer plants with lower operating costs. The hope for the AI at Big River is that it will lower operating costs and help to sell unused power when demand for electricity is high.

David Stickler, Chief Executive, Big River Steel Mill

One expert credited Big River for being at the cutting edge of steel mill technology. It is the world’s first steel plant designed to manage its operations with the aid of “artificial intelligence from the drawing board,” stated Ron Ashburn, executive director of the Association for Iron & Steel Technology.

Big Steel started the AI project in 2017, collecting and analyzing data and training algorithms used to predict maintenance requirements for new machinery. The system collects data on equipment conditions, assessing wear and tear in the hopes of reducing shutdown time and gaining operating hours.

Noodle.AI is also working with SSAB Americas, a global steel manufacturer, to pair the company’s Enterprise AI Platform with its sensor data with external data to help plan business operations, according to an account in Robotics Business Review. The plan is to improve machinery uptime, engage in predictive maintenance and seek ways to optimize the plant.

“We are excited to implement new digitalization technologies and to explore how the application of Enterprise AI can impact our performance and create a competitive advantage,” stated Tom Toner, Vice President of Operations for SSAB Americas. “Our goal is to learn how we can increase efficiency and decrease any bottlenecks in our operations with this advanced technology.”

Noodle.ai’s founder and CEO Steve Pratt stated, “SSAB Americas is a pioneering manufacturing company that is looking to embrace new technologies to improve the quality of their products, service to customers and competitiveness.”

The steel industry has seen disruption in the past two decades by steel plant capacity added in China, which now produces 50% of the world’s steel. As the Chinese began to export excess inventory at lower prices, it put pressure on western producers. As a result, steel manufacturers in the west are concentrating on improving efficiency by modernizing, according to an account written by Hiranmay Sarkar, a managing partner with Tata Consultancy Services, in SupplyChainBrain.

The steel making labor force has been reduced in favor of automation in the last 25 years, a period when world steel production grew by two and a half times, and the industry has reduced the workforce by more than 1.5 million members, Sarkar reported.

A digital twin is a digital replica of a physical asset, including its systems and devices. The twin can serve as the backbone for cyber-physical integration, enabling seamless transition of data between digital and physical worlds. To enable enterprise AI, Sarkar suggests, the digital twin needs to have these attributes:

  • An ecosystem commerce platform, off-the-shelf software, for information exchange with internal and external business partners;
  • Physical equipment connectivity and event capture, through IoT devices. This ensures real time data collection at various nodes of the supply chain, such as ore storage by miners, suppliers and vessel operators, production by coke oven, blast furnace and mill, product store and distribution by yards and freight transporters.

Read the source articles in WSJPro, Robotics Business Review and SupplyChainBrain.

Considering The Practical Impacts Of Achieving Einstein-Level AI

By Lance Eliot, the AI Trends Insider

Too smart for their own good.

Smarter than their britches.

Egghead.

Pointy head.

An Einstein.

These are the kinds of semi-polite insults that are sometimes used to take down someone that seems to be highly intelligent.

This can be especially used whenever the person evokes the know-it-all kind of stance and tries to lord over others with their professed smartness and smarty-pants attitude.

Not everyone that might be in the intellectual high-end rankings is necessarily the type that wants to make sure that you know they are the mental giant in the room, but it does seem to happen with great regularity and presumably to the delight of the brainy colossus that is overtly full of their own boastfulness.

How shall we weigh the brainiac in terms of gauging their peak-level intellectual power?

I suppose you could remove their brain, place it on a scale, and see how much it weighs.

Probably not very conducive though to their continuing capacity as a living, breathing, functioning human being. Speaking of physically measuring the brain, there have been all sorts of efforts to try to dig up brains of famously smart people and do various dissections of their brains, doing so in hopes of being able to ascertain what made them so sharp.

Nowadays, the usual measuring stick for figuring out someone’s intellectual proficiency is the IQ (Intelligent Quotient) test.

Using a standard such as the classic Stanford-Binet Intelligence Scale test, which was first promulgated in 1916, there are often published rankings that try to make claim to whom among us is the topmost intellect. Stephen Hawking was around an IQ score of 160, something that we know due to his actually having undertaken an IQ test.

Albert Einstein’s score of around 160 to 190 is an estimate based on analyses of his writings and works (he apparently never took an IQ test, though he could have done so, but perhaps opted purposely to not take it or never had cause to take one).

Typically, if you can score 115 or above you are labeled as someone with a high IQ.

Getting a score of over 132 will get you bumped-up into the highly gifted category. The 145 and above is considered at the genius level.

The highest ever recorded is supposedly a score of 263, but there is some disagreement about the matter (this score is attributed to Ainan Celeste Cawley, born in 1999 and alive to this day).

Questioning The IQ Measurement

Not everyone believes in the IQ bandwagon.

Some would say that the IQ test is a questionable means to measure someone’s intellectual prowess.

There are predetermined aspects such as the nature of your language, your culture, and your propensity to solve puzzles, all of which makes critics decry that the IQ number is at best a surrogate of intellect and at worst a misleading gauge of intellect. There are also concerns that those that perchance score high on the IQ test will then consider themselves a kind of special class of human, perhaps encouraging them to look down upon others. The Mensa group, which is a high-IQ association, admits only those that have at least a score of 132 or other such scores depending upon the IQ test being used.

Another qualm about IQ tests is that it seems to judge your bookworm kind of thinking, more so than a true “smartness” indicator.

I’m sure you’ve seen the common portrayal in movies and TV shows of the highly intellectual person that cannot tie their own shoes and cannot open a paper bag. If someone can do really well on tests that ask about obscure numeric patterns or mind-numbing word games, does this really showcase intellect? It might, depending upon your definition, but it generally is not considered the same as measuring your smartness.

Some believe that being smart is different from having a high intellect.

You might so happen to be highly intellectual and also highly smart. There are some that believe you can be highly smart, perhaps tip-top smart, and yet not necessarily have an extremely high intellect. Generally, the odds are that you’d score well on an IQ test, but the high IQ doesn’t necessarily translate into being highly smart, and nor does the aspect of being highly smart necessarily indicate you’ll be an A+ on an IQ test.

Another concern about any of the IQ tests is that your intellectual performance is being measured only at a given point in time.

Maybe at a relatively young age you could score a quite high IQ, but later on in your middle-aged years you aren’t able to score as high. Does that mean you’ve dropped in your intellect? This takes us into the other word that some like to use, the word is “wisdom” and for which once again there is a debate about the relationship between wisdom, intellect, and smartness.

You might gain wisdom as you grow older, at least that’s the usual expectation.

Will you also increase your IQ?

Some claim your IQ is your IQ, no matter what your age and when you perchance take an IQ test. This though does not bear out in terms of the reality, which is that people can take an IQ test at different points in their life and have differing scores. Plus, you can take a different kind of IQ test and score differently on it that you might on some other also “valid” IQ test.

The debate that really gets people bubbling on the IQ topic involves whether your IQ is based on nature versus nurture.

Are you born with a particular IQ level that will ultimately surface once you become of an age to be able to express it?

Thus, it’s a DNA kind of thing. Or, are we all perhaps born with the same IQ potential and your upbringing and environment will dictate how far your IQ will emerge? Perhaps it’s a nurturing element for which some of us happen to get the proper intellectual inspirational blooming and others of us don’t.

The half-in half-out answer is usually stated that you are born with some IQ capacity and it will either emerge or not depending upon your environment and how you are raised.

If we put a baby in the woods to be raised by wild wolves, and the baby happened to have an IQ of 260, which we had not yet been able to measure of the tiny tot but say we guessed that the tiny baby had such an IQ, would the genius level IQ ever be showcased? Would being among wolves allow for the IQ to come to the surface? Would a tree make a sound if it fell in the woods and there was no one around to hear it?

Darwin had an interesting take on intellect.

He proposed that your intellect might contribute toward your survivability in a manner you might not have previously considered. Sure, we would guess that if you had an IQ you could hopefully figure out how to make fire and hunt gazelles, which would presumably enhance your chances of survival. Darwin also hypothesized that topnotch intellect would attract mates and therefore boost your chance of survivability and for carrying on your legacy of high intellect.

For those of you that might have been beat-up by a strong-armed muscle rippling bully as a child, and for as much as our society seems to be keen on humans having muscular bodies, it is perhaps a surprise to consider that intellect might be so revered and be on Darwin’s favorites list.

We are used to the trope that the nerdish kid is the one that is physically meek and mild. The physical imposing one is the one that gets ahead and readily attracts all the mates. Our fascination with the character Spiderman is representative of this kind of imagery.

Learning From Parrots About IQ

A recent study of budgerigars, a type of parrot, provides an ingenious glimpse of how we might try to test Darwin’s hypothesis.

Researchers in China tried to construct an experiment to see whether female budgerigars would be more attracted to male budgerigars that demonstrated greater intellect than other male budgerigars involved in the study (this was research done by the Institute of Zoology at the Chinese Academy of Sciences in Beijing).

The male budgerigars were presented with a difficult foraging task. Some were shown how to solve it, but this happened outside the gaze of the female budgerigars.

The female budgerigars were able to watch the male participants try to open a container and access food. The males that had no prior training (i.e., not being shown the trick), were generally unable to open the container. The males that had the prior training could open the container. Presumably, the female budgerigars would infer that the males that were successful in getting the food were the intellectually sharper ones and the males that failed at doing so were intellectually inferior.

I’ll steer clear for now on the question of whether this is a gender-biased study and merely note it for your noteworthiness.

In any case, the outcome of the study was that the females tended to prefer the males that had succeeded in obtaining the food from the container. You might argue that it suggests the females were more attracted to the seemingly higher intellectual males. In a manner, it provides evidence to support Darwin’s hypothesis on the matter.

I realize that you are perhaps a bit skeptical about the experimental approach and whether the designed experiment really is on-target to Darwin’s theory.

For example, how do we know what the female budgerigars were really thinking about?

Maybe they ascribed other attributes to the males that succeeded in the task, and those attributes might have little or nothing to do with a perceived sense of intellectual prowess. Furthermore, the females were never allowed to try to undertake the task, so they were not fully aware of what the task consisted of and had to base their “choices” as to the males based solely on watching them perform the experimental task.

Another potential weakness about the study involves our overall conundrum about how to measure intellect. The means of figuring out how to get into a locked container might be considered a problem-solving kind of task, which might or might not require high intellect, and therefore we could debate if intellect is truly being encompassed and exhibited in this study. Were the males merely showcasing keen problem-solving skills rather than high intellect per se?

Based on the experimental design, we need to accept the idea that we are to infer that the container access matter is a sign of good problem solving and that correspondingly, a good problem solver is ergo a high intellectual. Recall that earlier it was pointed out that smartness and intellect are not necessarily the same. Why should we believe that keen problem-solving and intellect are necessarily the same?

They probably are not, most would likely say.

Is this problem-solving task a valid surrogate in lieu of administering to the budgerigars our now-accepted IQ measurement tool, namely the Stanford-Binet Intelligence Scale test?

Makes one kind of chuckle to consider how we might get the budgerigars to take a conventional IQ test. Ponder, how might we ask these Australian parakeets to take an IQ test. These gregarious parakeets are typically referred to as the budgie, and I’d suggest it would be quite interesting to watch as the budgie “read” a conventional IQ test and pencil in, or shall we say peck in, their answers.

Let’s get back to human intellect.

The parrot study was mainly to illuminate that intellect is presumably a quite important matter and that Darwin was a proponent of the belief that intellect ties to survivability, doing so for humans and other animals too.

Considering AI and IQ

In the field of Artificial Intelligence (AI), the presumed overarching goal consists of trying to make machines that seem to exhibit the equivalent of human intelligence.

I’ve tried to word that sentence carefully. Notice that I’m saying that the machine is not necessarily the same as humans in terms of how human intelligence exists.

Many would assert that if we can reach intelligence in machines and do so in a manner that might be quite different from how humans arise to intelligence, we have nonetheless succeeded in achieving artificial intelligence.

The famous Turing Test is a somewhat simple notion of how we might measure whether AI has been achieved or not. Generally, it consists of having a machine that has presumably AI that competes with a human that presumably has human intelligence, and another human asks questions of the two competitors. If the human inquisitor cannot differentiate between the two competitors and is unable to state which is the AI and which is the human, one could infer that the AI has achieved human intelligence.

For my detailed assessment of the Turing Test, see my article: https://www.aitrends.com/selfdrivingcars/turing-test-ai-self-driving-cars/

For the notion of so-called Super-Intelligence, see my article: https://www.aitrends.com/selfdrivingcars/super-intelligent-ai-paperclip-maximizer-conundrum-and-ai-self-driving-cars/

Whether we are facing a grand singularity, see my article: https://www.aitrends.com/selfdrivingcars/singularity-and-ai-self-driving-cars/

For my article about why some say we should start over on the AI pursuit, see: https://www.aitrends.com/selfdrivingcars/starting-over-on-ai-and-self-driving-cars/

For conspiracies about AI, see my article: https://www.aitrends.com/selfdrivingcars/conspiracy-theories-about-ai-self-driving-cars/

Here’s a good question to contemplate. How high is up?

I mention this because the question arises as to how much intelligence do we need to say that there is an AI that is indeed intelligent?

Suppose an AI system can pass the Turing Test.

Suppose further we give the AI an IQ test.

Many would claim that a score of 70 or lower is an indicator of an intellectual disability. Imagine what we would be pondering if the AI took an IQ test and got a score of say 50.

What a dilemma!

We have an AI system that appeared to pass the Turing Test and seems to be intelligent, and yet at the same time did quite poorly on the IQ test. I realize you might assert that the AI would have been unable to succeed at the Turing Test if it did not have a sufficient IQ, presumably an IQ of at least around 100, which is the “normal” average that usually is scored. I’m not so convinced that you are correct in that assertion.

I’ll shift our attention though from the bottom side of the IQ scale to the top side of the IQ scale.

How high up will we want the AI to score?

If the AI can score at say 115, which is the considered high-IQ range, would that be sufficient?

Consider this scenario.

Your life is in the hands of a robot that must decide what to do and potentially save you. You can choose a robot that has an IQ of 50 (considered intellectually disabled), or one that has an IQ of 100 (intellectually average for a human), or one that has a score of 115 (high IQ), or a score of 160 (Stephen Hawking’s score), or 190 (exceeds genius), or even let’s say the never-yet-human achieved score of 300 (knocking the socks off the IQ test!).

I’m guessing you’ll pick the highest possible number.

You would presumably use the logic that the higher the intellect of the robot then the greater the chance of it making sure your life is saved. Why take a chance on a robot that has “only” an IQ of 160 (Hawking’s level and Einstein’s level), if you could pick one that is off-the-charts at 300? If you could get yourself a robot that had the AI equivalence of two-times the score of Einstein, it would seem unwise of you to take anything lower.

Right now, AI systems are being built and deployed, but there isn’t anyone especially measuring what their intellectual score is. The belief seems to be that if the AI can “do the job” it was intended to do, hopefully it is intellectually commensurate enough. Should we be pleased with this approach? Are you willing to be at the mercy of an AI system for which no one even knows how intellectually low or high it is?

We also need to revisit the earlier points about smartness versus intellect.

I can tell you straight out that the AI of today does not have smartness.

The AI of today is brittle and considered narrow and lacks what often is referred to as Artificial General Intelligence (AGI).

I also earlier mentioned the notion of wisdom, which, again the AI of today would be far below any kind of wisdom scale (not even anywhere on such a scale). There are ongoing efforts to try to imbue AI with common sense reasoning, but it is a long slow road, and nobody knows whether it will ever even succeed.

For my assessment of common sense reasoning efforts, see my article: https://www.aitrends.com/selfdrivingcars/common-sense-reasoning-and-ai-self-driving-cars/

For plasticity in Deep Learning, see my article: https://www.aitrends.com/ai-insider/plasticity-in-deep-learning-dynamic-adaptations-for-ai-self-driving-cars/

For the boundaries of AI, see my article: https://www.aitrends.com/selfdrivingcars/ai-boundaries-and-self-driving-cars-the-driving-controls-debate/

For the AI irreproducibility problem, see my article: https://www.aitrends.com/selfdrivingcars/irreproducibility-and-ai-self-driving-cars/

AI Self-Driving Cars and IQ

What does this have to do with AI self-driving driverless autonomous cars?

At the Cybernetic AI Self-Driving Car Institute, we are developing AI software for self-driving cars. One question that nobody seems to yet be asking is whether we are supposed to be aiming for regular cognition or something more pronounced such as superior cognition. This ties to the discussion herein so far about intellect.

Allow me to elaborate.

I’d like to first clarify and introduce the notion that there are varying levels of AI self-driving cars. The topmost level is considered Level 5. A Level 5 self-driving car is one that is being driven by the AI and there is no human driver involved. For the design of Level 5 self-driving cars, the automakers are even removing the gas pedal, brake pedal, and steering wheel, since those are contraptions used by human drivers. The Level 5 self-driving car is not being driven by a human and nor is there an expectation that a human driver will be present in the self-driving car. It’s all on the shoulders of the AI to drive the car.

For self-driving cars less than a Level 5 and Level 4, there must be a human driver present in the car. The human driver is currently considered the responsible party for the acts of the car. The AI and the human driver are co-sharing the driving task. In spite of this co-sharing, the human is supposed to remain fully immersed into the driving task and be ready at all times to perform the driving task. I’ve repeatedly warned about the dangers of this co-sharing arrangement and predicted it will produce many untoward results.

For my overall framework about AI self-driving cars, see my article: https://aitrends.com/selfdrivingcars/framework-ai-self-driving-driverless-cars-big-picture/

For the levels of self-driving cars, see my article: https://aitrends.com/selfdrivingcars/richter-scale-levels-self-driving-cars/

For why AI Level 5 self-driving cars are like a moonshot, see my article: https://aitrends.com/selfdrivingcars/self-driving-car-mother-ai-projects-moonshot/

For the dangers of co-sharing the driving task, see my article: https://aitrends.com/selfdrivingcars/human-back-up-drivers-for-ai-self-driving-cars/

Let’s focus herein on the true Level 5 self-driving car. Much of the comments apply to the less than Level 5 self-driving cars too, but the fully autonomous AI self-driving car will receive the most attention in this discussion.

Here’s the usual steps involved in the AI driving task:

  • Sensor data collection and interpretation
  • Sensor fusion
  • Virtual world model updating
  • AI action planning
  • Car controls command issuance

Another key aspect of AI self-driving cars is that they will be driving on our roadways in the midst of human driven cars too. There are some pundits of AI self-driving cars that continually refer to a Utopian world in which there are only AI self-driving cars on the public roads. Currently, there are about 250+ million conventional cars in the United States alone, and those cars are not going to magically disappear or become true Level 5 AI self-driving cars overnight.

Indeed, the use of human driven cars will last for many years, likely many decades, and the advent of AI self-driving cars will occur while there are still human driven cars on the roads. This is a crucial point since this means that the AI of self-driving cars needs to be able to contend with not just other AI self-driving cars, but also contend with human driven cars. It is easy to envision a simplistic and rather unrealistic world in which all AI self-driving cars are politely interacting with each other and being civil about roadway interactions. That’s not what is going to be happening for the foreseeable future. AI self-driving cars and human driven cars will need to be able to cope with each other.

For my article about the grand convergence that has led us to this moment in time, see: https://aitrends.com/selfdrivingcars/grand-convergence-explains-rise-self-driving-cars/

See my article about the ethical dilemmas facing AI self-driving cars: https://aitrends.com/selfdrivingcars/ethically-ambiguous-self-driving-cars/

For potential regulations about AI self-driving cars, see my article: https://aitrends.com/selfdrivingcars/assessing-federal-regulations-self-driving-cars-house-bill-passed/

For my predictions about AI self-driving cars for the 2020s, 2030s, and 2040s, see my article: https://aitrends.com/selfdrivingcars/gen-z-and-the-fate-of-ai-self-driving-cars/

Returning to the topic of cognition and intellect, let’s consider how the matter of the level of intellect applies to the advent of AI self-driving cars.

We’ve so far considered whether there is a need to aim for a “highest feasible” intellect for an AI system that we might be constructing and fielding.

For AI that is designed and built to drive a car, what level of intellectual prowess should be the overarching goal?

First, you could say that we should aim at the level of intellect as exhibited by humans in the case of performing the driving task.

That would seem to be a reasonable marker as to the intellect that we as a society expect for execution of driving a car.

In that case, you would be hard-pressed to suggest that any kind of “higher” intellect is needed per se. Generally, the average person is able to obtain a driver’s license and legally be able to drive a car. As such, we’d presumably say that an “average” IQ is sufficient for the driving effort, and therefore we could be satisfied with an average IQ in terms of the AI that would be driving a car. Perhaps a score of around 100 would be satisfactory.

Suppose we pushed to get the AI of a self-driving car to a higher level of IQ.

Would we gain much?

It is not especially convincing that a higher intellect is going to make that much difference in undertaking the driving task. Are expert-level drivers that race cars of a higher intellect? There doesn’t seem to be much study on that matter, but I’d guess that those race car drivers are more versed in the driving of cars and yet are not intellectually especially at a higher level than the rest of us. Are professional drivers such as cabbies or truck drivers at a higher level of intellect than average car drivers? Again, there doesn’t seem to be much evidence to suggest they are.

If we don’t seem to have a base of high intellects that drive cars, in other words no set of high IQ’s that happen to drive cars and that have been studied to see whether they are somehow more proficient at driving cars, we are left to speculate about the higher IQ and its relationship to driving. You could claim that a higher intellect might be able to think more rapidly when driving a car and be able to mentally add something to the driving chore.

Perhaps a higher intellect would allow a human driver to be more adept at piecing together the clues of the driving scene.

They might be able to see that there is a car up ahead and that there is a pedestrian on the sidewalk, and be able to put together puzzle pieces in a manner that lets them know the odds are that the car is going to hit its brakes, due to the pedestrian likely stepping onto the street, which will then cause the cars behind the stopped car to come to a sudden halt, and will cascade into a potential car crash. Notably, all of these mentally complex calculations being undertaken in a fraction of second, faster and more completely than someone of a lesser but average intellect.

In that manner, a higher intellect might foster being able to envision more complex car-related traffic possibilities. A higher intellect might enable the driver to find clues about the driving situation that those of an average intellect would fail to piece together. A higher intellect might suggest that the driver would be faster at processing the driving situation. This faster mental processing might allow for being able to sooner avoid adverse driving moments. Whereas an average driver might get “caught off-guard” because of not having detective-like realized the clues of a pending driving problem, a higher intellect might be more likely to do so. And by mentally processing it faster, this gives the higher intellect driver more available options since they sooner ascertained that some driving action was needed, upping the chances of being able to select among more early escape options.

For my article about the speed of cognition aspects, see: https://www.aitrends.com/selfdrivingcars/cognitive-timing-for-ai-self-driving-cars/

For the role of defensive driving mental calculations, see my article: https://www.aitrends.com/selfdrivingcars/art-defensive-driving-key-self-driving-car-success/

For the human foibles of driving, see my article: https://www.aitrends.com/selfdrivingcars/ten-human-driving-foibles-self-driving-car-deep-learning-counter-tactics/

For the driving complexities, see my article: https://www.aitrends.com/selfdrivingcars/prevalence-induced-behavior-and-ai-self-driving-cars/

For my article about scene analysis, see: https://www.aitrends.com/selfdrivingcars/street-scene-free-space-detection-self-driving-cars-road-ahead/

Will Higher Intellect Boost Driving

I realize you might argue that perhaps the higher intellect is not necessarily going to get all of those driving advantages.

Similar to the study of the budgerigars, perhaps driving a car is a problem-solving task and not as influenced simply by having higher intellect. You could assert that being able to perceive a driving scene and make life-critical decisions about operating a car is more so a problem-solving task rather than a purely intellectual exercise.

Thus, we might be barking up a wrong tree by trying to lay claim that the higher intellect will ergo lead to being a more adept driver.

The higher intellect might allow someone to be a better or faster problem-solver, but this is not axiomatic. These are two different items, whether being a topnotch problem solver versus having a high intellect. Presumably, if a higher intellect wanted to be a topnotch problem solver, they might have an easier time of doing so, prodded on by their high intellect, though it is not automatically the case.

We can also wonder whether a higher intellect might actually work against the notion of being a better driver of a car.

Remember the earlier mention that we as a society seem to assume that the higher intellect is often in the clouds in terms of not paying attention to day-to-day elements of life. We portray high intellects as unable to tie their own shoes. If that’s the case, it would seem that suggesting they are going to be driving a car at a higher plateau of driving proficiency is actually the opposite of what we should expect. We apparently should be worried about these higher intellects driving a car. They might be less able to do so in comparison to an average intellect driver.

Why would it be the case that a higher intellect might be a poorer or worse driver than someone of an average IQ?

You might at first assume that certainly the higher intellect would win at any task involving intellectual effort. The physical aspects of driving are generally rather simplistic, involving pushing a brake pedal and an accelerator pedal, and steering a wheel, all of which even a very young child can do. It’s the intellectual aspects of driving a car that appear to make the difference of being a proficient driver versus one that is not so proficient. A driver that cannot think quickly enough and tie together their sensory clues is one that is seemingly more likely to get into car accidents and create untoward traffic conditions.

We already as a society are concerned about distracted drivers. A distracted driver is one that is not paying attention to the driving task. The distraction can involve a physical form of distraction, such as taking your hands off the wheel to manipulate your smartphone, or maybe turning your head to talk to someone in the backseat of the car and thus your head is now turned away from the driving scene. The distraction can also be a form of intellectual distraction.

When your mind is focused on a text that you have just read on your smartphone, you are no longer well-engaged in the driving task. Even if your head and eyes are now facing the roadway, your mental awareness of the traffic conditions is going to be weakened by your mental preoccupation with the text that you read. I know that there is a lot of concern about using a smartphone while driving, but we’ve already had other forms of mental distractions too, such as talking with others in your car and discussing say the latest in politics or some other non-driving related matter.

You don’t even necessarily need to have something prompt you to mentally become disengaged with the driving task. Have you ever caught yourself daydreaming while driving your car? Imagine you are driving from Los Angeles to San Francisco, a six hour or so drive, and suppose it is a quiet traffic day and the main highway is pretty much empty. Nothing but miles upon miles of farms and rolling hills. For some people, they find themselves unable to concentrate on the roadway and their minds wander. This lack of mental connection to the driving task can catch them unaware if suddenly a tire blows or a deer darts across the highway.

One could suggest that at a higher level of intellect you might be able to multitask mentally more so than someone of an average intellect. If that’s the case, perhaps a minor mental distraction would not materially impact your driving, while for the person of average intellect it could have a more pronounced impact. In essence, if we imagine that intellect is like an apple pie, thinking about some text that you just got might consume half of the apple pie for an average intellect, but only a tiny slice of the apple pie of the higher intellect.

On the other hand, one could claim that perhaps the greater intellect is more prone to tossing their intellect at everything that comes along. In that case, whereas the average intellect might devote just a small mental slice to consider the text they just received, it could be that the higher intellect pours all of their mental capacity into thinking about the text, therefore having very limited intellect leftover to focus on the driving task.

For tests about human responsiveness while driving, see my article: https://www.aitrends.com/selfdrivingcars/not-fast-enough-human-factors-ai-self-driving-cars-control-transitions/

When humans get themselves into a tit-for-tat while driving, see my article: https://www.aitrends.com/selfdrivingcars/tit-for-tat-and-ai-self-driving-cars/

For my article about the role of greed, see: https://www.aitrends.com/selfdrivingcars/selfishness-self-driving-cars-ai-greed-good/

For my article about the dangers facing back-up drivers, see: https://www.aitrends.com/selfdrivingcars/human-back-up-drivers-for-ai-self-driving-cars/

Being Overly Smart Has Potential Downsides

I had earlier indicated that we often say that someone is smarter than their britches or too smart for their own good.

If we reword this to suggest that someone has too high an intellect for their own good, let’s see how that might impact their intellectual prowess and see how it could impact their driving.

I’ll consider these five exemplars of the potential adverse consequences of high intellect:

  • Analysis Paralysis
  • Dismissiveness
  • Shallowness of Thought
  • Over-Thinking
  • False Over-Confidence

Analysis Paralysis.

A higher intellect might be more prone to analyzing a myriad of options. Will that car ahead opt to make a sudden lane change? Will the pedestrian leap into the street? Is that traffic light going to change to red in the next few seconds? All of this thinking can produce analysis paralysis. The driver becomes preoccupied with analyzing what to do or what might happen, and as a result they aren’t making the kinds of rapid decisions that need to be made when driving a car.

Dismissiveness.

A higher intellect might be dismissive of others. You’ve likely had someone that thinks they are so sharp that they dismiss other people’s ideas or suggestions. Unless they believe the other person is of an equal intellect, they don’t get much credence to the other person. A driver that is dismissiveness might opt to ignore a warning from a front seat passenger that tells them a car to their right is possibly going to intervene into their lane. This dismissiveness can undermine the driving effort.

Shallowness of Thought.

A higher intellect will often categorize mental tasks and then proclaim that a particular task is not worthy of their intellectual powers. As a driver, a higher intellect might be tempted to consider the driving task as menial. As a result, the person is unwilling to put much mental effort toward driving. They prefer to operate a car with a shallowness of thought. If they do so, it could spell danger as they are potentially underestimating what they need to be considering in order to be a safe driver.

Over-Thinking.

A higher intellect might tend toward over-thinking every moment of the driving task. I knew someone that was looking at every angle at every step of driving a car. They made incredible mental leaps about the aspects that could go awry, almost to the degree that they even were calculating the chances of a meteor striking the earth in front of their car. This over-thinking can cause them to become muddled and overwhelmed about the driving task.

False Overconfidence.

A higher intellect might believe that they are the best driver ever, which is fueled by their belief in their own astounding intellect. This leads to overconfidence. They assume that for any driving situation they will be able to mentally find a means of driving the car to escape. This type of driver can be riskier in their driving and get themselves into binds that they are actually unable to get out of safely.

I am not saying that only higher intellects will potentially fall victim to the aforementioned mental guffaws. Any driver can suffer from analysis paralysis, and from dismissiveness, and from shallowness of thought, and from over-thinking, and from false over-confidence. I’d bet though that the higher intellect is perhaps more likely to find themselves falling into these traps. It is the basis for why we have as a society come up with the too smart for their own britches label.

Could an AI system for a self-driving car also be vulnerable to these same kinds of mental underpinnings?

Sure, each of these intellectual dangers can readily happen to an AI system. I don’t want you to though assume I am saying that AI is sentient, and it is succumbing to these mental impairments in the same manner that a human might. I am not suggesting or implying this.

Instead, I am trying to assert that the AI as a form of automation can suffer the same deficiencies and it is up to the AI developers to try to make sure that the AI is not caught off-guard by these computationally equivalent mental maladies.

For example, analysis paralysis can befall the AI if it gets bogged down trying to explore a large search space and fails to realize that time is crucial to making a driving decision. The AI could be so engrossed in assessing the sensory data and the virtual world model that it lets the clock continue to run. The running clock means that the world outside the self-driving car is moving and changing, which might mean that the AI is gradually losing out on options for making a vital car driving decision.

I had predicted that the Uber incident in Arizona might partially have occurred because of the time taken by the AI to try to assess the driving situation. Preliminary reports assessing the Uber incident appeared to have echoed that point. Though some might shrug their shoulders and say that’s just the way the real-time automation works, I am not one to fall into the trap of allowing automation to be some kind of independent amorphous entity that happens to do what it does. I hold accountable the AI developers that should be developing their AI systems to handle these kinds of real-time situations.

For my initial assessment of the Uber incident, see: https://www.aitrends.com/selfdrivingcars/initial-forensic-analysis/

For my follow-up about the Uber incident, see my article: https://www.aitrends.com/selfdrivingcars/ntsb-releases-initial-report-on-fatal-uber-pedestrian-crash-dr-lance-eliot-seen-as-prescient/

For the responsibility aspects of AI self-driving cars, see my article: https://www.aitrends.com/selfdrivingcars/responsibility-and-ai-self-driving-cars/

For my article about egocentric AI developers, see: https://www.aitrends.com/selfdrivingcars/egocentric-design-and-ai-self-driving-cars/

For my article about AI developers and groupthink dangers, see: https://www.aitrends.com/selfdrivingcars/groupthink-dilemmas-for-developing-ai-self-driving-cars/

Conclusion

Is superior cognition needed to drive a car?

We might debate the meaning of the word “superior” and be at odds about the notion of what being superior in cognition consists of.

If we use the everyday notion of IQ, the question can be rephrased as to whether higher IQ is needed to drive a car. There seems little evidence to suggest that any particular level of above average IQ is a needed element to drive a car, since the world at large appears to be able to drive a car, and we can reasonably assume therefore it involves an average IQ effort.

It could be that if we can achieve AI that can drive a self-driving car, we might want to see what it can do if it is pushed to a higher level of intellect. Perhaps we might have better driving and safer driving. This is not necessarily the case. We also need to be aware of the kinds of mental maladies that seem to at times correspond to having higher intellect, and whether those might be found in AI systems and therefore undermine the heightened intellect aspects.

I’ve not entertained herein the conspiracy theorists that are worried that we might be pushing the AI intellect to a point that it surpasses human intellect and then opts to take over humanity. The paperclip making super-intelligence mankind-overtaking AI I’ve covered elsewhere. For now, I’m merely trying to get AI developers to consider the degree of intellect that they are aiming to achieve in their AI systems, and also prodding the rest of us to also consider what level of intellect are we becoming vulnerable to in terms of AI systems that increasingly are entering into our lives.

I’ve highlighted the nature of AI self-driving cars as a key indicator of how the intellect might come to play. Many AI systems are not as involved in making immediate life-or-death decisions as those of AI self-driving cars. I would hope that we would be more concerned about the intellect prowess of AI systems in the role of deciding whether a multi-ton car is going to make a right turn or maybe come to a sudden stop, decisions on which the lives of humans hang in the balance. It sure seems like having superior cognition would be a handy capability, if properly designed and deployed.

The Einstein AI for self-driving cars has kind of a ring to it, doesn’t it.

Copyright 2020 Dr. Lance Eliot

This content is originally posted on AI Trends.

[Ed. Note: For reader’s interested in Dr. Eliot’s ongoing business analyses about the advent of self-driving cars, see his online Forbes column: https://forbes.com/sites/lanceeliot/]

Financial AI Is the Missing Key to Ending Human Trafficking

By Caleb Danziger

Technology has opened up a world of possibility, for good and for bad. Some enable criminals to operate on a level previously unseen, but the solution to stopping them often also lies in tech. At today’s rapid pace of development, catching the bad guys is usually a matter of having the most advanced tools.

Some of the most pressing criminal concerns have moved to the digital sphere. As a result, cyber justice can take the form of a technological arms race. Clever implementation of technology may have allowed criminals to avoid capture in the first place, but as security tech improves, it may prove to be their downfall.

Human trafficking is one crime that has proved historically challenging to address. Those guilty of this heinous activity have repeatedly slipped the grasp of law enforcement, but thanks to new tools like artificial intelligence (AI), that may be changing.

A Growing Global Concern

Also called modern slavery, human trafficking is a widespread and grave issue. An estimated 40.3 million people are victims of the practice, roughly a quarter of whom are children. While most of it may happen without people realizing it, it takes place in every corner of the globe, even in the United States.

Unfortunately, human trafficking has been able to grow because of technological advances. One of the leading reasons behind this growth is modern financial technology. Digital forms of payment have enabled traffickers to launder illegal funds with greater ease and security, making them harder to track through their money.

Modern slavery has gotten to be such a massive industry that it brought in an estimated $150 billion in 2018. You might think revenue of that size would be easy to track, but thanks to the variety of digital finance options, it’s proven nearly impossible. Under-the-radar finances aren’t the only way technology has helped these criminals evade capture, either.

Difficulty in Helping Trafficking Victims

One of the largest obstacles to helping trafficking victims is identifying and then finding them. Traffickers are careful and discreet in their operations, but a significant factor is simply the modern digital environment. The online world is so enormous that criminals can easily hide in it.

Creator of anti-trafficking app Traffic Jam, Emily Kennedy stated that with traditional tracking methods, “massive amounts of data online actually made it harder to locate victims.” It may seem that a larger data pool would make tracking is easier, but it proved to be too much information to sort through.

Emily Kennedy, Traffic Jam creator

Finding relevant information in the vast oceans of online data can be like finding a needle in a haystack. Sorting through all of it could take substantial amounts of time — which officials could use doing more practical work. Every minute spent looking through this data is another minute human trafficking victims are still at risk.

How AI Is Fighting Human Trafficking

Thankfully, an answer has emerged. The surplus of data that once hid human traffickers is now what could lead to their identification and capture, thanks to AI. An older system may not be able to make sense of all this information, but it’s an asset to AI.

Machine learning systems can analyze massive quantities of information in record time. As these systems grow closer to strong AI, programs that can think for themselves, and they become all the more useful. Advanced AI systems can train themselves to be better at identifying specific trends and commonalities.

In her experience using AI to help trafficking victims, Kennedy said she found “computer visions can identify the same pattern in many different photos.” Using this technology, she and her colleagues were able to help law enforcement find victims that traffickers had advertised in the same hotel room.

This same computer vision tech allows AI to recognize faces, even if they’ve changed over time. Kennedy and law enforcement officials found they could use her program Face Search to find victims based on old photos. While human officers couldn’t recognize victims after aging and changes to their hair and makeup, the AI program could.

Financial AI Technology

Just as financial technology made it easier for traffickers to get away, AI helps out them. Machine learning programs can find connections between seemingly unrelated accounts and transactions to identify money laundering. The more cases these systems identify, the better they’ll become at recognizing them.

Finding suspicious behavior between scattered accounts can be time-consuming or near-impossible for traditional algorithms, and even more so for humans. An AI devoted to the process can spot red flags faster and more accurately. With more data, they could improve even further.

Organizations like the UK’s Financial Conduct Authority have recognized the potential for financial AI in fighting crime. Consequently, there has been a recent push toward unified action. If banks and legal institutions share their data, these account-tracking AI systems could substantially strengthen the fight against human trafficking.

Healthcare Providers Beginning to Apply AI More in Patient Care

By AI Trends Staff

Hospitals and doctors’ offices collect vast amounts of data on their patients, everything from blood pressure to genetic sequencing. While the data may be digitized, using it to help in patient treatment can be challenging. But the healthcare industry is getting better at using AI to find patterns in data that can help in patient care.

“I think the average patient or future patient is already being touched by AI in health care. They’re just not necessarily aware of it,” stated Chris Coburn, chief innovation officer for Partners HealthCare System, a hospital and physicians network based in Boston, in an account in WebMD. The application of AI to patient care is in an early stage and is spreading.

“I could not easily name a [health] field that doesn’t have some active work as it relates to AI,” stated Coburn, who mentioned pathology, radiology, spinal surgery, cardiac surgery, and dermatology as examples.

And of course entrepreneurs see a business opportunity. GNS Healthcare of Cambridge, Mass., offers a causal machine learning and simulation technology, that combined with its reach into “next-generation” patient data, can help determine which blood cancer patients are likely to gain the most from bone marrow transplants. The company has found a genetic signature in some multiple myeloma patients that suggested they would benefit from a transplant.

“We now have the data, the technology and the processing speed to build disease models and run computer-based in silicon simulations on every possible treatment scenario and inform the physician on the right treatment for each individual based on their biology. That’s the real power of AI,” stated GNS Healthcare co-founder Iya Kahlil. Dr. Kahlil is a physicist who co-invented the computation engine underlying products of GNS Healthcare and Via Science. Her expertise spans applications in drug discovery, drug development, and treatment algorithms that can be applied at the point of care.

Dr. Iya Kahlil, co founder, GNS Healthcare

Among the challenges for increased use of AI in medicine are requirements to keep patient data private, while processing huge volumes of data. While names can be removed from large data sets, people today can be identified by their genetic code, noted Mike Nohaile, senior vice president of strategy, commercialization, and innovation for Amgen, the pharmaceutical giant.

Doctors also have to guard against racial and demographic bias in the data; it’s difficult to understand and interpret the algorithms running the AI. “I don’t want to trust a black box to make decisions because I don’t know if it’s been biased,” stated Dr. Nohaile. “We think about that a lot.”

AI in Clinical Diagnoses

AI algorithms are able to diagnose diseases faster and more accurately than doctors; in particular, AI is good at detecting diseases from image-based test results, writes Terence Mills, CEO of AI.io, a data science and engineering company, writing recently in Forbes. On its website, AI.io describes its products as putting “white box AI in a nutshell” and, “If white box AI is a well-constructed house, black box AI is the foundation & framing.”

Late last year, Google’s DeepMind was able to train a neural network to accurately detect over 50 types of eye diseases by analyzing 3D rental scans, showing how effective AI technology can be at identifying real anomalies.

Early detection is key to the effective treatment of cancer, such as with preemptive measures. Certain types of cancer, such as different types of melanoma, are very difficult to detect during the early stages. AI algorithms are able to scan and analyze biopsy images and MRI scans 1,000 times faster than doctors, with an 87% accuracy rate.

Precision Medication

Medication is ideally dispensed with precision according to the correct treatment for the patient’s diagnosis. Precision medicine depends on the interpretation of vast volumes of data.

The patient’s data, including treatment history, restrictions, hereditary traits, and lifestyle, is used to determine the most effective medication. This organization of data is a strong suit for machine learning and AI algorithms. AI data management systems are able to store and organize large volumes of data to draw meaningful conclusions and make predictions.

AI systems can browse through archives of patient data stores by hospitals and health care facilities, to assist doctors in formulating precision medication for individual patients. AI prescription systems can study the patient’s medical history and help determine the likelihood that the patient will take the medication as prescribed.

Read the source articles in  WebMD and Forbes.

7 Aspects that Make the Cloud a Safer Place for your Data

An important decision all organizations need to make regarding their data is whether to store it on-premise or to host it in the cloud. As of 2019, more than 94 % of them have found the answer to this question and rely on the cloud for storing their data. Others are still questioning the impact that such a decision will have on their business model. But one of the most frequent questions asked relates to data security: “What about the security of our data in the cloud?”Below is a list of 7 aspects that make cloud storage more secure than data warehousing centers: Data Redundancy & Crash ResilienceLet’s start with the obvious: the safety of your data relates directly to the safety of the physical data-center(s) where it is stored.Since cloud vendors store data redundantly, multiple copies of your files reside on a number of data centers spread across the globe. This gives you the certainty that in case of the worst failure (e.g. a physical server is destroyed by a natural disaster) your data is still safe and available. This makes it much easier to recover from disasters.On the other hand, on-premise data hosting is much riskier from this ...


Read More on Datafloq