Where Does Encryption Fit in Privacy Regulations?

Organizations today view data as an asset. In fact, most companies pride themselves on the data they have. Yet at the same time, global privacy regulations have put strict rules on how organizations store and keep secure customers' data.

According to a recent study by IDC, by 2023, people will create nearly 102.6 zettabytes of data every year. Data volumes like this may sound good, but leave consumers open to a broader array of cybercrimes and make organizations vulnerable as well. Organizations are stepping up their data encryption practices in an effort to make it safer for the data they have stored and to reduce the risk of data sprawl.

Data Encryption Under the GDPR

The General Data Protection Regulation (GDPR) is the largest data privacy regulation in the world and is currently viewed as a base standard. The GDPR recognizes encryption as an important part of ensuring data privacy, which is why under article 32, "security of processing"  the GDPR states:

“Taking into account the state of the art, the costs of implementation and the nature, scope, context and purposes of processing, as well as the risk of varying likelihood and severity for the rights and freedoms of natural persons, the controller and ...


Read More on Datafloq

Understanding BDaaS and Its Types

The essence of Big Data lies within its definition of deploying high volumes of data, velocity along with the variety of data sets that can become quite difficult for extracting value and managing them. Most of the businesses today have now started gaining insights on how Big Data is going to possess challenges and opportunities for them at the same time. However, the increased adaptability of Big Data has opened several options for the organizations that are looking to capture, manage and analyze large data volumes that can accelerate the decision-making processes and ultimately create competitive advantages. As mentioned before, big data does bring in several challenges as well. Businesses involved with Big Data have started recognizing that they do suffer from an inadequate capacity for processing and storing their data. This, however, is not a clear indication for the businesses as it reflects that the companies cannot deploy the existing large volumes of data to the maximum. So, to overcome all the challenges and encash on the opportunities, businesses have started incorporating Big Data as a Service or BDaaS. Defining BDaaSBDaaS is a framework that comprises of cloud-based and distributed technologies, allowing the users to work efficiently with the ...


Read More on Datafloq

Why You Should Prioritize Data Transformation Above Other Digital Transformation Initiatives

Chance are you’re aiming to invest in a BI and analytics program to capitalize on the big data your company has been acquiring over the years. But before you spend millions on opting for expensive BI programs, take a step back and ask yourself three questions:Do I have data I can trust?Do I understand my data?Do I have a data transformation & data quality framework in place?A, ‘No’ to these questions indicates that you need to optimize your data before you invest in a BI or analytics program. And this piece will help you understand how.A Few Statistics to Push You into Action Here are statistics from a survey conducted by the HBR in determining why most companies are failing in their data-driven efforts. The survey shows:72% of survey participants report that they have yet to forge a data culture69% report that they have not created a data-driven organization53% state that they are not yet treating data as a business asset52% admit that they are not competing on data and analyticsAlarming figures? In our experience of working with 4,500+ clients from across the globe, we know all too well the truth behind these statistics.Organizations are ramping up their effort to ...


Read More on Datafloq

Creative People Using AI to Explore New Territory

By AI Trends Staff

Human thought has always been central to creativity. This has been true through development of printing presses, gramophones, cameras, camcorders, typewriters, word processors, photo editing software and many other tools invented over centuries.

Maybe AI changes the game, suggests a recent account in TechTalks based on a reading of “The Artist in the Machine: The World of AI-Powered Creativity,”  by Arthur I. Miller. Not that the book asserts AI will replace human creativity, but that AI is bringing change to the creative arts.

Key advances include: AI-assisted art, including an application called style transfer. Well-trained neural networks map the style of one image onto another. First proposed in 2015 by Leon Gatys in a paper titled, “A Neural Algorithm of Artistic Style.” It allows for example a photograph to take on the style of a van Gogh painting. Gatys is affiliated with the University of Tuebingen, Germany.

Leon Gatys, author of “A Neural Algorithm of Artistic Style,” affiliated with the University of Tuebingen, Germany

Style transfer has caught on, finding commercial applications in social media platforms. “I want to have a machine that perceives the world in a similar way as we do, then to use that machine to create something that is exciting to us,” Gatys is quoted as Miller’s book.

Another innovation is Pix2Pix, an AI algorithm that can convert a rough sketch into a real photograph. The Dutch broadcasting network NPO developed Pix2Pix as part of a project to use AI to analyze human creations and turn them into lifelike paintings. It uses a specialized form of generative adversarial network (GAN), which have been used in many creative AI projects, including creation of a painting that sold for $432,000.

“Pix2Pix empowers people who may not have the requisite motor skills and technical skills to express their creativity,” stated Phillip Isola, the creator of Pix2Pix. “It allows mixing of science and art together, offering a means to show data in a way that’s provocative, emotional, and compelling.”

Read about how to use Pix2Pix at tom’s guide.

Inventors Need to Be People in Europe

The discussion of whether AI will replace human creativity continues. The European Patent Office recently turned down an application that described a food container, because it was created by AI. The applicable law says inventors need to be actual people, according to an account in Fast Company written by Tim Schweisfurth, associate professor for Technology and Innovation Management, University of Southern Denmark, and René Chester Goduscheit, professor of Technology and Innovation studies, Aarhus University, Denmark.

The authors outline how AI is being used in the creative process, notably in GANs applied to pictures. “But even if machines can create innovations from data, this does not mean that they are likely to steal all the spark of human creativity any time soon,” the authors state. “Even if machines cannot replace humans in the creative domain, they are a great help to complement human creativity.”

The use of AI in the creative process is called “innovation analytics” by authors of a recent account in ScienceDirect, who also see AI in a support and not a replacement role. “Extant literature coupled with our experiences as practitioners suggest that while AI may not be ready to completely take over highly creative tasks within the innovation process, it shows promise as a significant support to innovation managers,” the authors state. They describe computer-enabled, data-driven insights, models and visualizations as innovation analytics. “AI can play a key role in the innovation process by driving multiple aspects of innovation analytics,” they state.

Chinmay Kakatkar from Ludwig Maximilian University of Munich, and a senior data scientist at Fineway of Munich, a firm working on smart travel, was lead author of the paper.

Dr. Chinmay Kakatkar, Data Scientist, Fineway

Poetry and Physics Interact with Quantum Computing

The interaction of poetry and physics was the pursuit of poet Amy Catanzano when she created “World Lines: A Quantum Supercomputer Poem.” which translates the quantum theory behind a topological quantum computer in both its word choices and its visual structure, a practice Catanzano calls quantum poetics. “My aim was to write a poem that served as an imaginative and rigorous site of interaction between poetry and physics,” she stated in a recent account in Physics.

She describes poetry as a nuanced and complex form of language that goes beyond simple dictionary definitions of individual words. Poems use rhythm, visual structure, line breaks, word order, and other devices to explore invisible worlds, alter the flow of time, and depict the otherwise unimaginable, states Catanzano, who is also an assistant professor of English at Wake Forest University in Winston-Salem, N.C..

World Lines is happening in phases, with phase one nearly complete. She is writing more quantum supercomputer poems in phase 2, and in phase 3, she is working to bring World Lines into a 3D environment and art installation. In August 2019 on a visit to CERN, she spoke with Joao Pequenão, head of the MediaLab at CERN and a multimedia storyteller, about her goal. He suggested using gaming technology, AI, machine learning and virtual reality software to bring the poem into a 3D environment.

Amy Catanzano, poet, and assistant professor of English, Wake Forest University

While phase 1 of World Lines is complete, I am writing other quantum supercomputer poems in Phase 2. For phase 3, I am taking steps to work with scientists to bring World Lines into a 3D environment and art installation. In August 2019, during my second site visit to CERN, I spoke once again with Joao Pequenão, head of the MediaLab at CERN and a multimedia storyteller, about my goal. He suggested the possibilities of using gaming technology, artificial intelligence, machine learning, and virtual reality software to bring the poem into a 3D environment.

“I imagine an environment through which the reader moves, writing the poem as they walk,” Catanzano stated, in the hopes that poetry can help physicists develop a more effective language to describe the complex ideas of quantum physics.

Artist Mario Klingerman, who has used Pix2Pix to transform portraits into award-winning paintings, sees machines with AI as having a better opportunity than humans to create. Humans build on what they have learned but machines can create from scratch, he suggests, stating, “I hope machines will have a rather different sort of creativity and open up different doors.”

Read the source articles in TechTalks, Fast Company and in Physics.

AI Tools Will Help Us Make the Most of Spatial Biology

Contributed Commentary by David W. Craig, Ph.D. and Brooke Hjelm, Ph.D.

We have heard a lot about cellular and tissue spatial biology lately, and for good reason. Tissues are heterogeneous mixtures of cells; this is particularly important in disease. Cells are also the foundational unit of life, and they are shaped by those cells proximal to them. Not surprisingly, the research field sought to survey cellular and tissue heterogeneity. The last decade saw massive adoption of single-cell sequencing RNA. This approach requires that we disaggregate cells, leading to accounting and characterization of cell populations, but at the same time losing their spatial context such as their proximity to other cells or where they fit with traditional approaches such as histopathology.

Enter Spatial Genomics

That’s why we have welcomed spatial transcriptomics and a focus on mapping RNA transcripts to their location within a tissue. After all, understanding disease pathology requires that we understand not only the underlying genomics and transcriptomics but also the relationship between cells and their relative locations within a tissue. Along for the ride: new avenues for the study of cancer, immunology, and neurology, among many others. What’s changed is the emergence of new tools for resolving spatial heterogeneity. SeqFISH and MerFISH are novel approaches for mapping gene expression within model systems. Multiple companies such as 10x Genomics and NanoString are now democratizing access to spatial transcriptomics, introducing new technologies and assays. They are opening up the study of disease pathology.

AI & Deep Learning: Adding to Our Vocabulary

New experimental methods often start with historical analysis approaches. Let’s consider the first step in analysis: finding clusters of spots/cells with similar gene expression and then visualizing by reducing dimensions. In single-cell RNA-seq, the tSNE projection and color-coding clustering may be the signature plot, much like the Manhattan plot was to the GWAS.

Yet, critically, we haven’t leveraged the underlying histopathology image—the foundation of diagnosis and study of disease. We haven’t leveraged the fact that two spots are neighboring. What happens when we do? What happens at the edges between two clusters? What happens when cell types intersperse or infiltrate, such as in immune response? Are there image analysis methods we aren’t considering that have a high potential impact?

Indeed, concepts such as convolutional neural networks (CNNs) and generative adversarial networks (GANs) have been instrumental in classifying features and underlying hidden layers. We can go beyond the tSNE in spatial transcriptomics—and the question should be about viewing the latent space (the representation of the data that drives classifying regions and the discovery of hidden biology). These terms and concepts are foundational when it comes to artificial intelligence and need to be front and center in spatial transcriptomics analysis.

Of course, the use of AI and deep learning terminology is ubiquitous. Getting away from the hype, from self-driving cars to the successes in image recognition (ImageNet Challenge), some of the most remarkable achievements leverage spatial and imaging data. Data matters and one then asks: should we consider a single spatial transcriptomics section as one experimental data point, or is it 4,000 images and 4,000 transcriptomes?

In spatial biology, we can anticipate that applying AI to cell-by-cell maps of gene or protein activity will pave the way for significant discoveries that we might never achieve on our own. Incorporating spatially-resolved data could be the next leap forward in our understanding of biology. There will be questions we never even knew to ask that may be answered by combining spatial transcriptomics and spatial proteomics. But to get there, we need to come together and work as a community to build up the training data sets and other resources that will be essential for giving AI the best chance at success.

We have yet to truly make the most of the spatial biology data that has been generated. If we do not address this limitation, we will continue to miss out even as we produce more and more of this information.

David W. Craig, PhD (davidwcr@usc.edu), and Brooke Hjelm, Ph.D. (bhjelm@usc.edu) are faculty within the Department of Translational Genomics, University of Southern California Keck School of Medicine.

Google Rules AI, with TensorFlow at Foundation, Leadership in Core Products

By John P. Desmond, AI Trends Editor

The way Google came from nowhere with the launch of Android in 2007 to today dominating the smartphone operating system market, is what the company is doing now with AI, some market observers suggest.

Google now has an 80 percent share of the worldwide smartphone OS market, and it has seeded the AI market by making its TensorFlow software library open source, putting it at the foundation of many AI applications, suggests a recent account in Analytics Insight.

Some 50 Google products use TensorFlow to build deep learning applications to help differentiate companions in Photos to refinements in the core search engine. Google has become a machine learning organization.

The authors state, “Google has gone through the most recent three years constructing a gigantic platform for artificial intelligence and now they’re unleashing it on the world.”

Differential Privacy Library Aims to Enhance Personal Security

Among recent releases is a “differential privacy library” – used to protect personal data while scanning massive volumes of data. Google wants to engage the development community in this new discussion about privacy protection. The “differential” works to cryptographically mask private information while drawing information from datasets.

By publicly releasing the library on GitHub, development companies with startup assets can explore a rigorous approach to privacy. Healthcare could be an interested segment.

“Differential security is a high-affirmation, analytic means of ensuring that use cases are addressed in a privacy-preserving manner,” stated Miguel Guevara, a Google product manager in the privacy and protection office, in a blog post. Health care researchers may for example want to look at the average amount of time patients spend at different clinics, to see if there are differences in care.

Miguel Guevara, leading product development for differential privacy, Google

AI Can Be Expensive

This work costs some money. A joint project of Carnegie Mellon University and Google to build XLNet, a new language model, generated a discussion about how much the services cost. Eliot Turner, entrepreneur, AI expert and now co-founder of Hologram AI, estimated that it cost the university $245,000 for 2.5 days to train the XLNet model. That was based on a resource breakdown that he outlined, according to an account in Medium.

That number was challenged by Google researchers who specified different resources and arrived at an estimate of $61,440 for 2.5 days. Also likely is the Google team did not charge full price, since it was leading the project.

XLNet was said to outperform the previous state-of-the-art (SoTA) for language tasks, called BERT (Bidirectional Encoder Representations from Transformers). XLNet achieved SoTA results on 18 of 20 language tasks. The model is big, thus expensive to run.

In another project, the University of Washington and the Allen Institute for AI in May 2019 developed Grover, a 1.5-billion-parameter neural net, tailored to detect fake news. Recently outsourced to Github, training for the Grover model cost a reported $25,000.

The GPT-2 language model recently developed by OpenAI, demonstrates impressive performance across a range of language tasks, such as machine translation, question answering, reading comprehension and summarization. The computer power required to run the model for training costs $256/hour.

Many machine learning models are running on smaller footprints. Computer scientist Yoshua Bengio, Turing Award winner and scientific director of MILA (Montreal Institute of Learning Algorithm), was quoted as saying, “Some models are so big that even in MILA (Montreal Institute of Learning Algorithm) we can’t run them because we don’t have the infrastructure for that. Only a few companies can run these very big models they’re talking about.”

In a recent financial report on Google’s business from its parent, Alphabet, notes that Google’s core products such as Search, Android, Maps, Chrome, YouTube, Google Play and Gmail each have over one billion monthly active users. “We believe we are just beginning to scratch the surface,” the report stated, in an account from Strategic Management Insight.

Google is a leader in acquisitions as well, making 118 acquisitions between 2012 and 2015, far outpacing Microsoft, Facebook and Apple.

Google’s revenue is generated by performance and brand advertising, and machine learning and AI are driving the company’s latest innovations, the report notes.

Google sees challenges to its business coming from general purpose search engines (Baidu, Bing, Yahoo), vertical search engines and e-commerce websites (Amazon, eBay, LinkedIn), social networks (Facebook, Twitter); providers of digital video services, enterprise cloud services and digital assistant providers.

One of the best sources of information about what is happening with AI at Google is of course Google itself. The Google AIBlog for example offers accounts of research written by participating engineers.

In one example, A Scalable Approach to Reducing Gender Bias in Google Translate, describes a project to provide gender-specific translations, male and female. The approach was tried with Turkish-to-English, and has recently been expanded to English-to-Spanish.

“We’ve made significant progress since our initial launch by increasing the quality of gender-specific translations,” the researchers stated on the blog, adding, “We are committed to further addressing gender bias in Google Translate and plan to extend this work to document-level translation, as well.”

Read the source articles in Analytics Insight,  Medium, Strategic Management Insight the Google AIBlog.

Rewilding of AI Autonomous Cars: The Final Era

By Lance Eliot, the AI Trends Insider

The great outdoors. You might assume that there are remote forests that are still pristine and untouched by humanity. If you aren’t trained as a botanist or biologist or ecologist, you might not be aware that many of these seemingly unspoiled forested lands are actually quite marred by the hands of mankind.

In some areas, there is a concerted effort to reinstate the earlier status quo of those lands. This involves not only protecting what is there, but also includes doing a systematic restoration to the wilderness too.

There are specialists that refer to this as wildlife reengineering.

A wildlife engineer studies the existing state of the ecosystem and tries to devise a means to re-introduce wildlife into it. The goal is to do this in a fashion that the ecosystem ultimately becomes self-regulating and self-sustaining. Mankind tries to push it toward a true wildlife wilderness and then hopefully steps aside and doesn’t need to continually be in the middle of doing so (other than providing further protection from mankind itself).

There’s a term that has arisen for this process of re-instituting the wilds to a wilderness state, namely it is called rewilding.

Rewilding As A Controversial Matter

Keep in mind that not everyone believes in the notion of rewilding.

Some opponents say that mankind is part of nature and so it is already “natural” that the ecosystem has changed because of mankind’s presence.

Trying to somehow restore the wild to a pre-mankind arrival doesn’t make much sense to them. Even if you could do such a restoration, are you then going to ban humans from going into these restored wild areas? Denying human access seems like a rather nutty option, in their view.

Proponents toward rewilding tend to say that they are not trying to remove mankind from the equation, and only trying to undo the damages that mankind has wrought.

They say that we all now know more about what kind of adverse impacts the introduction of mankind can have. This will allow for the revived ecosystem to co-exist with mankind, in which mankind now takes better care as to how the interaction with the wilderness takes place. Few of the proponents say mankind should be banned and instead argue that there should instead be limits and regulations on what mankind can do in the wild, trying to in essence save mankind’s base instincts from itself.

You might have seen some of the wildlife overpass crossings that are put up over highways that run through a wilderness area.

This is an example, some say, of a rewilding tactic.

The wildlife overpass crossing is made to be as natural and nature-like as feasible, including having trees, plants, grass, and so on. Animals that might have otherwise tried to cross the actual highway, and be struck by human’s driving their cars, will hopefully tend to use the specially constructed overpass crossing instead.

Notice in this example that the highway was not removed from the wilderness, though of course some rewilders would want that aspect to occur. Instead a kind of “compromise” was found that aids the wildlife and still aids mankind. One could also say that the overpass saves human lives. It presumably is a win-win, allowing for some amount of rewilding and yet also retaining mankind’s interest in the wilderness area.

As in any endeavor, there are some proponents of rewilding that are at the extreme end and would say that the use of an overpass is not even considered a form of rewilding. To them, true rewilding would consist of routing that highway to avoid the wilderness entirely. One question that arises by those that are sympathetic to such notions and yet also have some qualms, involves the cost for doing such rewilding. A large-scale full-scale rewilding of an enormous wilderness is not going to be cheap. Where will the money come from to fund such an enterprise?

Proponents of rewilding would say that we all owe the wilderness for the prior transgressions of our ancestors. Like it or not, we need to now share the cost to put the wild back into the wilderness. And though I previously mentioned forests, the “wild” refers to any kind of biome or habitat, whether it is a forest, a desert, and so on. Some consider conservationists to be rewilders, while others suggest that you can be a conservationist and not necessarily be a rewilder — or at least have varying views on the topic of rewilding.

What does this have to do with AI self-driving driverless autonomous cars?

At the Cybernetic AI Self-Driving Car Institute, we are developing AI software for self-driving cars. Our view is that there are several eras underlying the evolution of AI self-driving cars, and one of those eras will involve the “rewilding” of AI self-driving cars.

Allow me to elaborate.

I’d like to first clarify and introduce the notion that there are varying levels of AI self-driving cars. The topmost level is considered Level 5. A Level 5 self-driving car is one that is being driven by the AI and there is no human driver involved. For the design of Level 5 self-driving cars, the automakers are even removing the gas pedal, brake pedal, and steering wheel, since those are contraptions used by human drivers. The Level 5 self-driving car is not being driven by a human and nor is there an expectation that a human driver will be present in the self-driving car. It’s all on the shoulders of the AI to drive the car.

For self-driving cars less than a Level 5 or Level 4, there must be a human driver present in the car. The human driver is currently considered the responsible party for the acts of the car. The AI and the human driver are co-sharing the driving task. In spite of this co-sharing, the human is supposed to remain fully immersed into the driving task and be ready at all times to perform the driving task. I’ve repeatedly warned about the dangers of this co-sharing arrangement and predicted it will produce many untoward results.

For my overall framework about AI self-driving cars, see my article: https://aitrends.com/selfdrivingcars/framework-ai-self-driving-driverless-cars-big-picture/

For the levels of self-driving cars, see my article: https://aitrends.com/selfdrivingcars/richter-scale-levels-self-driving-cars/

For why AI Level 5 self-driving cars are like a moonshot, see my article: https://aitrends.com/selfdrivingcars/self-driving-car-mother-ai-projects-moonshot/

For the dangers of co-sharing the driving task, see my article: https://aitrends.com/selfdrivingcars/human-back-up-drivers-for-ai-self-driving-cars/

Let’s focus herein on the true Level 5 self-driving car. Much of the comments apply to the less than Level 5 self-driving cars too, but the fully autonomous AI self-driving car will receive the most attention in this discussion.

Here’s the usual steps involved in the AI driving task:

• Sensor data collection and interpretation
• Sensor fusion
• Virtual world model updating
• AI action planning
• Car controls command issuance

Up until current times, most of the work on AI self-driving cars has been primarily research oriented, often done by university research labs and other government related entities. Some would say this was the “pre-historic” era of AI self-driving cars, though I think using such language as “pre-historic” is a disservice to those that have done incredible work to get us to where we are today.

Today’s era might be characterized as the Exploratory era.

As a result of a grand convergence of key factors, the advent of AI self-driving cars is beginning to emerge. It isn’t going to just suddenly up and appear. Instead, it will be years of trial efforts. I’ve mentioned many times that there will be hard times ahead for AI self-driving cars during this experimentation and exploratory period.

Will the public be willing to endure the hardships that will occur as AI self-driving cars get involved in untoward incidents on our roadways?

Will they tolerate such instances, or will it draw their ire and might there be a backlash that generates onerous regulations that dampen the efforts toward true Level 5 self-driving cars?

For my article about the grand convergence that has led us to this moment in time, see: https://aitrends.com/selfdrivingcars/grand-convergence-explains-rise-self-driving-cars/

See my article about the ethical dilemmas facing AI self-driving cars: https://aitrends.com/selfdrivingcars/ethically-ambiguous-self-driving-cars/

For potential regulations about AI self-driving cars, see my article: https://aitrends.com/selfdrivingcars/assessing-federal-regulations-self-driving-cars-house-bill-passed/

For my predictions about AI self-driving cars for the 2020s, 2030s, and 2040s, see my article: https://aitrends.com/selfdrivingcars/gen-z-and-the-fate-of-ai-self-driving-cars/

Next Comes The Taming Era

I’ve predicted previously that we’ll make it through the Exploratory Era, and we’ll then move into the Taming Era.

During the Taming era, there will be a socio-technical interplay of how the auto makers and tech firms devise and shape the AI of self-driving cars, along with the public and regulators, doing so to arrive at a “tamed” version of Level 4 and Level 5 self-driving cars.

Numerous hearings will take place.

Advocacy groups that are today rather subdued on the matter will come forth and be quite outspoken.

There will be sides taken as to pro-AI self-driving cars versus opposed to AI self-driving cars.

By this somewhat acrimonious process, there will be a kind of “taming” of AI self-driving cars.

By the word taming I’m referring to the kinds of driving capabilities that the AI self-driving cars will embody.

Right now, most of the automakers and tech firms are creating rather timid AI drivers. In some sense, you might liken this to a novice teenage driver. When a teenage driver first starts to drive a car, they usually tend to come to a stop at a stop sign and take a seemingly long time to look around before they proceed. They tend to drive at or below the speed limit. You’ve perhaps found yourself behind one of these novice drivers and gotten frustrated that they are driving with such a sluggish pace and in a rather timid manner.

For AI self-driving cars, given the rudimentary and rather crude capabilities of today’s versions, it certainly makes sense that the AI will be driving in a similar timid manner. The public is already on-edge about whether to trust AI self-driving cars. It won’t take too many incidents of an AI self-driving car that injures or kills someone, whether it be human occupants in the self-driving car or humans outside of the self-driving car, and the public support for AI self-driving cars will wane quickly.

For the public’s perception of AI self-driving cars, see my article: https://aitrends.com/selfdrivingcars/roller-coaster-public-perception-ai-self-driving-cars/

For the Uber self-driving car incident that killed a pedestrian, see my analysis: https://aitrends.com/selfdrivingcars/ntsb-releases-initial-report-on-fatal-uber-pedestrian-crash-dr-lance-eliot-seen-as-prescient/

For rear-end collisions and AI self-driving cars, see my analysis: https://aitrends.com/ai-insider/rear-end-collisions-and-ai-self-driving-cars-plus-apple-lexus-incident/

So, we’ll migrate from the Exploratory era to the Taming era, which will achieve finding a balance between having AI self-driving cars on our roadways and have them function in a manner that is more akin to a refined novice driver rather than a more seasoned sophisticated driver.

Some believe for example that an AI self-driving car should always drive in a purely legal manner. This idea that AI self-driving cars would need to strictly adhere to the legal requirements for driving a car will inhibit the kind of driving that we already accept by humans. In essence, if the AI is going to only be able to drive in a strictly legal fashion, you are by definition going to limit the savviness of the driving by the AI.

For the case of allowing AI to not be bound strictly, see my article: https://aitrends.com/selfdrivingcars/illegal-driving-self-driving-cars/

For the human foibles of driving, see my article: https://aitrends.com/selfdrivingcars/ten-human-driving-foibles-self-driving-car-deep-learning-counter-tactics/

For how AI developers are designing the AI, see my article: https://aitrends.com/selfdrivingcars/egocentric-design-and-ai-self-driving-cars/

For the nature of driving styles for AI self-driving cars, see my article: https://aitrends.com/selfdrivingcars/driving-styles-and-ai-self-driving-cars/

I’d anticipate that we’ll have reached a point that the public generally accepts the use of AI self-driving cars on our roadways.

People will find these tamed AI self-driving cars to be reasonably acceptable for use among the general public. It will be as though we’ve all agreed to let teenagers drive cars and even though we’ll realize they aren’t going to improve much over time, at least they are consistent in the tamed nature of how they are driving. Being a predictable driver is handy, even if the driver is timid and tentative as to how they drive.

When I mention that the AI self-driving car won’t improve much over time, I am not saying that this is due to the limits of the AI per se. Instead, I am suggesting that since the tamed or timid nature of the AI has been arrived at as a societal preference, the boundaries will be established not so much by what the AI can actually do, but more so by what society is willing to allow for the AI to do.

Don’t though overstate that aspect, since I am not saying that the AI will be able to drive miraculously, and it is somehow artificially being held back. There will be a crossover point of the further maturation of the AI and the limits imposed by society as part of the taming period.

Keep in mind too that there will be a mix of both human driven cars and AI self-driving cars during this period of time. I realize that there are pundits that say we will only have AI self-driving cars, but this belies the aspect that today we have about 200+ million cars in the United States, and there is nothing that will somehow overnight change those into AI self-driving cars.

For many years, there will be a mix of human driven cars and AI self-driving cars. I point this out because the Utopians envision a world in which cars are all communicating and coordinating with each other, and there is zero chance of fatalities due to automobile incidents. It really doesn’t make much sense to be discussing this all-and-only AI self-driving cars world since it is a far distant future, and instead be more practical and face the reality of a world composed of both human driven cars and AI self-driving cars.

What kind of driving won’t the tamed AI do?

Imagine you are on the freeway. If you are a human driver, and you are heading to work for the day, you might be willing to weave in-and-out of the traffic, trying to make headway in an otherwise somewhat crowded freeway.

The AI of the self-driving cars for the Tamed Era will instead tend to stay in their lane, and only make a lane change when actually required. Thus, the AI made a lane change to get onto the freeway, and maybe made another lane change to the fast lane since the distance to work is far enough to warrant getting into the fast lane. Once the exit nears, the AI instructs the self-driving car to move over lanes to make the exit.

For the limited self-driving car maneuverability of today’s AI, see my article: https://aitrends.com/selfdrivingcars/maneuverability-ai-self-driving-cars/

For understanding that humans drive and make use of greed, see my article: https://aitrends.com/selfdrivingcars/selfishness-self-driving-cars-ai-greed-good/

For defensive driving tactics and AI self-driving cars, see my article: https://aitrends.com/selfdrivingcars/art-defensive-driving-key-self-driving-car-success/

In that sense, the AI is being “held back” from driving in a manner that a human would drive.

Presumably, if the AI is as good as a human driver, it should be able to weave in-and-out of the traffic, just as a human would. I realize there are risks associated with this kind of driving tactic, and I’m not suggesting that the weaving approach is necessarily good, but on the other hand it does have some advantages and provides a driving technique that when used sensibly can be helpful.

With today’s AI, it would be prudent to not trust a self-driving car to do this kind of weaving tactic. The AI is just not good enough as yet. It would be similar to telling a teenage novice driver to weave through traffic on the freeway. I think we would all shudder at the prospects of such an act. The odds are that the weaving would as a minimum disturb the rest of the traffic, and more than likely lead to a car incident of one kind or another.

Eventually, the AI is going to be good enough to do this kind of weaving.

But, we as a society will not be ready for it and will instead need to undergo the Taming Era to have sufficient belief and faith that AI self-driving cars can at least handle the rudiments of driving on our roadways.

It is at this juncture that the third era will arise, the Rewilding Era.

The Final Era: Rewilding

I’ve opted to call this third and final era the “rewilding” period because in some sense we are going to be returning AI self-driving cars back to what was originally envisioned.

It was envisioned that AI self-driving cars would drive as proficiently as humans.

This includes the various day-to-day tricks that we humans use.

It includes the borderline legal maneuvers that humans use when driving.

It involves perhaps some degree of illegal driving that humans use today.

And so on.

For my article about the notion of starting over on AI for self-driving cars, see: https://aitrends.com/selfdrivingcars/starting-over-on-ai-and-self-driving-cars/

For the cognition timing aspects of the driving task, see my article: https://aitrends.com/selfdrivingcars/cognitive-timing-for-ai-self-driving-cars/

For the responsible parties involved in the AI self-driving car realm, see my article: https://aitrends.com/selfdrivingcars/responsibility-and-ai-self-driving-cars/

We’ll shift from the tamed AI self-driving car to the rewilded AI self-driving car.

I’m not going to say that the AI self-driving cars will be “wild” per se, and indeed I don’t think we would want them to be driving in a crazy or foolish or senseless way. Instead, I’m suggesting that we’ll take off the earlier imposed limits and allow the AI to flourish and drive the self-driving car in a manner more consistent with the “wild” way in which humans drive a car.

In a manner of speaking, you might say that we’ll be putting a tiger-in-the-tank of AI self-driving cars.

For those of you that don’t recall, one of history’s most well-known advertising campaigns occurred during the 1960’s and involved a slogan that you should put a tiger in your tank. It became a quite popular saying and did well for the Esso gasoline provider.

Eventually, the phrase itself became commonplace as an expression of getting energized.

Conclusion

Here then are the three eras of AI self-driving cars:

1)      Exploratory Era (today)

2)      Taming Era (coming up)

3)      Rewilding Era (post-Taming)

We will progress from today’s Exploratory Era into the Taming Era.

This will allow us as a society to generally become accepting of AI self-driving cars. Once that has occurred, there will be an impetus to shift toward the third era, the Rewilding Era. At that juncture, the AI will be sophisticated enough that it will be time to take-off the training wheels, so to speak, and let it roll.

I’d say it is premature right now to be telling people that someday we’ll be putting a tiger in their tank and making AI self-driving cars become aggressive human-like drivers.

Unless you’ve first lived in the Taming Era, there is little chance you can readily perceive that there’s a further future ahead for AI self-driving cars that involves such driving tactics.

We’ll need to ease our way into it.

That’s fine, though, and provides breathing room for the AI to get better suited for driving, and become in some sense just like those maniac human “driving tigers” that we all are.

Copyright 2020 Dr. Lance Eliot

This content is originally posted on AI Trends.

[Ed. Note: For reader’s interested in Dr. Eliot’s ongoing business analyses about the advent of self-driving cars, see his online Forbes column: https://forbes.com/sites/lanceeliot/]

Social Listening Tools Powered by AI Going for Deep Personalization

By AI Trends Staff

Social listening tools powered with AI are becoming a powerful way to measure customer sentiment and conduct audience research. These tools are good at mining unstructured text, such as in social media posts, and taking measurements. Brands use them to track, analyze and respond to conversations about them on social media.

“The combination of data analytics, A.I. and social media affords us the ability to deeply and rapidly analyze customer opinions. Trends and patterns appear and enable comprehensive market research into key consumer insights,” states Sarah Lim in an account on the blog of  Remesh, which offers a platform to support products, campaigns and brands through research.

Social listening helps to discover customer insights and see what the customers value. The insights help to build the relationship with the customer, offer relevant product recommendations and increase sales. One objective is to know what the customers want before they do. Another goal is to reach a deeper level of customer engagement.

One startup is Xineoh, offering to predict consumer behavior with AI. The company positions to offer the same technology Amazon and Netflix have employed to help predict what products consumers would buy. The platform finds patterns in historical consumer data, applies those to current customer data, then issues recommendations. Xineoh’s founders have a background in the application of mathematical modeling and machine learning to advertising technology.  Based in Johannesburg, South Africa, the company was founded in 2015 and has raised $3.5 million, according to Crunchbase.

Writing in the Hootsuite blog, content marketing consultant Tony Tran defines social listening as when you track your social media platforms for mentions and conversations related to your brand, and then analyze them for insights and opportunities to act.

Tony Tran, content marketing consultant

He suggests a two-step process: first, monitor social media channels for mentions of your brands, competitors, products and keywords related to your business; second, analyze the information for ways to take action, from responding to a happy customer to shifting your entire brand positioning.

Social “listening” is distinguished from social “monitoring” in that social monitoring looks at metrics while social listening adds an action component that responds to the data, Tran suggests.

“Social listening looks beyond the numbers to consider the mood behind the data,” he states; some refer to this as “social media sentiment.”

He offered five tips for social listening: listen everywhere; learn from the competition; collaborate with other teams; roll with the changes; take action.

Hootsuite offers its own social listening tool for setting up a listening post; it can also be paired with resources from BrandWatch, a digital consumer intelligence company, to access 1.3 trillion social posts in real time.

Here is a look at selected companies offering social listening products or services.

Digimind Offers AI-Powered Social Media Listening

Digimind describes its services as being AI-powered social media listening, providing real-time software to assist brands to listen, engage, analyze, and report, according to an account from Influencer Marketing Hub. The tool displays social conversations about the customer’s brand, splitting them into categories and giving each a sentiment rating of positive, negative or neutral.

Digimind Social also allows you to see your brand image based on Google search behavior. You can follow in real-time what your customers want to know about your brands, products, people, and those of your competitors.

The company launched Digimind AI Sense in late 2018, resulting from joint research work between Digimind Labs and IBM Research teams. The tool was a response to markets who expressed interest in using AI to deliver better customer experiences, to analyze a lot of data and achieve highly-granular personalization as a result.

Digimind AI Sense was said to leverage machine learning, Natural Language Processing and image recognition. It offered automated tagging of mentions, relevance scoring, sentiment analysis based on thousands of qualified mentions, and image recognition trained on thousands of qualified images. This would allow users to see text mentions of their brands with logo presentations as well.

Synthesio Offers Social Media Intelligence

Synthesio positions as offering social media intelligence plus next-generation AI. The firm’s Social Media Intelligence Suite monitors the online presence of the business and provides insight. The platform captures social media information from 195 companies in more than 80 languages, and captures sentiment analysis in more than 20 languages, according to the company. The listening platform can be customized and integrated into a dashboard, along  with paywall data from LexisNexis, logo recognition and consumer reviews.

The Synthesio platform can track views, likes, favorites, replies, retweets, and shares from Facebook, Twitter, Instagram, and YouTube directly onto the listening dashboards. Users can add high-level business intelligence with Media Value and Engagement Rate widgets.

Massive pre-filtered data sets can be exported via API or using an in-house tool; social listening metrics can be merged with performance data in many SaaS business intelligence tools.

The company announced its Signal trend detection and insight module in 2019, a result of a collaboration with its parent Ipsos, which has years of experience with data science and statistics. The two companies outlined plans to bring new products and features to market as a result of significant investments in AI, R&D staff, data sources, integrations, images and video analysis, sentiment analysis and user experience.

Read the source articles at Remesh, Hootsuite and Influencer Marketing Hub.

What Will Future Battery Technology Look Like?

Batteries are a vital part of modern-day life. They provide power and energy for things like phones and remotes, but can also power much bigger things like cars or even airplanes. Innovation has brought them to new levels, but future battery technology will go even further. Batteries will likely face challenges as they advance. However, they are moving in the right direction, including sulfur and hydrogen models. Sustainability is a necessity, and the following battery innovations are possible now through technology. Whether or not they come to fruition is up to investors, consumers and industry leaders. SulfurLithium-ion batteries are the standard for most items. However, lithium-sulfur batteries offer a better alternative. They are lighter and have more energy density, and sulfur is more widely available and cost-efficient. Due to its advantages, some believe it will eventually overtake lithium-ion. Researchers are testing lithium-sulfur batteries and attempting to make them last longer. Since some models' cathodes tend to fall apart after 50 cycles, experts must perfect the battery before it's ready for implementation. However, with its advantages, it is a strong contender for replacing lithium-ion in the future. SodiumSodium-ion batteries seek to replace the lithium part of the compound. This abundant element ...


Read More on Datafloq

How Machine Learning for Fraud Detection Works in Different Industries

In a world where there are no absolute security guarantees, any organization could potentially become a victim of fraud. And non-profit companies, too - because data sometimes takes on even more weight than pure money. The more companies come up with protection methods, the more scammers come up with ways to outwit them in return. This is like running in a vicious circle, and the only thing that can break this circle is the ability to predict fraudulent attempts and prevent them. Machine learning is capable of this, and in this article, we will look at how fraud detection with ML works for different industries.

Fraud Detection with ML for Different Industries

Every year, a business loses up to five percent of profits due to fraud. It may seem that this is a very small amount, which is not worth worrying about, but in monetary terms, this figure can reach almost four billion dollars. That is, four billion is the net income of fraudsters, which very often is reinvested in other illegal operations. This situation clearly requires a response.

“Businesses can no longer afford to leave machine learning out of their fraud detection arsenal,” said Ashley Kramer, SVP Product Management at Alteryx.

In practice, ...


Read More on Datafloq

Docker images for agents: New names and What's next

We would like to announce the renaming of the official Docker images for Jenkins agents. It does not have any immediate impact on Jenkins users, but they are expected to gradually upgrade their instances. This article provides information about the new official names, upgrade procedure, and the support policy for the old images. We will also talk about what’s next for the Docker packaging in Jenkins.

New image names

• jenkins/agent is the new name of the old jenkins/slave image, starting from 4.3-2

• jenkins/inbound-agent is the new name of the jenkins/jnlp-slave image, starting from 4.3-2

• jenkins/agent is the new name of the old jenkins/ssh-slave image, starting from 2.0.0

See the upgrade guidelines below.

Why?

The "slave" term is widely considered inappropriate in open source communities. It has been officially deprecated in Jenkins 2.0 in 2016, but there are remaining usages in some Jenkins components. The JENKINS-42816: Slave to Agent renaming leftovers EPIC tracks cleanup of such usages. Official Docker agent images were a glaring case, it was not easy to fix that with the previous versions of the image release Pipelines on DockerHub. It is great to have the image naming issue finally fixed by this update.

Another notable change is replacing the JNLP agent term with inbound agent. Historically "JNLP" has been used as a name of Remoting protocols. JNLP stands for Java Network Launch Protocol which is a part of the Java Web Start. Jenkins supports Java Web Start mode for agents when running agents on Java 1.8, but our networking protocols are based on TCP and have nothing to do with Java Network Launch Protocol. This name has been very confusing since the beginning and became worse with the introduction of WebSocket support in Jenkins 2.217 (JEP-222). Docker agent images support WebSockets, so we decided to change the image name to jenkins/inbound-agent so that it prevents further confusion. Inbound agent term refers to agent protocols in which the agent initiates the connection to the Jenkins master through different protocols.

Thanks a lot to Alex Earl and krufab for the repository restructuring groundwork which made the renaming possible! Also thanks to Tim Jacomb, Marky Jackson, Mark Waite, Ivan Fernandez Calvo and other contributors for their reviews and testing.

Upgrading and Compatibility Notes

Good news, there are no breaking changes caused by this renaming. All images have been already modified to use the new terminology internally. If you use the recent versions of the previous images, you can just replace the old names with the new ones. These names may be referenced in your Dockerfiles, scripts, and Jenkins configurations.

We will keep updating the old images on DockerHub for at least 3 months (until August 05, 2020). There will be no new configurations and platforms added to the old images, but all existing ones will remain available (Debian for Java 1.8 and 11, Alpine for Java 1.8, etc.). After August 05, 2020, the old images will no longer receive updates, but previous versions will remain available to users on Dockerhub.

What’s next?

We will continue renaming of the Docker images in Jenkins components which reference old image names. There is also a set of convenience Docker images which include build tools like Maven or Gradle which will be renamed later. The jenkins/ssh-agent image might be renamed again in the future as well; see the ongoing discussion in this developer mailing list thread.

If you are rather interested in new features in Jenkins Docker packaging, stay tuned for future announcements! There are multiple ongoing initiatives which you can find on the public Jenkins roadmap (in the draft stage, see JEP-14). Some stories:

• General availability of Windows images.

• Support for more platforms (AArch64, IBM s390x, PowerPC).

• Switching to AdoptOpenJDK.

• Introducing multi-platform Docker images.

If you are interested in any of these projects and would like to contribute, please reach out to the Platform Special Interest Group which coordinates initiatives related to Jenkins in Docker.

Regarding the agent terminology cleanup outside Docker images, we will keep working on this project in the Advocacy & Outreach SIG. If you see the usage of the obsolete "slave" term anywhere in the Jenkins organization (Web UI, documentation, etc.), please feel free to submit a pull request or to report an issue in the JENKINS-42816: Slave to Agent renaming leftovers EPIC. There are "just" 3000 occurences left in the jenkinsci GitHub organization, but we will get there. Any contributions will be appreciated!

Tips for effective negotiation by video conference

We are dealing with unprecedented events, but we still need to step up, adapt and deliver. During this time when we are unable to meet face-to-face, we have a responsibility to secure our businesses and ensure deliverables for society to continue. Negotiation itself will still need to continue; in fact, it may even increase in the short-term as we undertake renegotiation of things like terms and conditions in order to maintain business continuity. With technology as it is, videoconferencing (VC) is already being used widely. But now it’s an essential everyday tool, so let’s consider how we can use this medium more effectively when negotiating. Not everybody is a lover of the video and many people leave their cameras turned off; yet as trained, skilled negotiators we understand the benefit of communication and the level to which non-verbal communication (NVC) plays its part in ensuring appropriate outcomes. Like everything else in life practice make perfect, and effective use of visual technology requires practice and attention. So, remember two things:· You need to read their body language· You need to send the messages you want the other party to ...


Read More on Datafloq

Govt eases norms to fly drones, exempts UIN, operator permit for some firms

The pilots will be part of the Directorate General of Civil Aviation's beyond visual line of sight experimental drone operations

How to build a Quality of Service (QoS) analytics solution for streaming video services

Customer Satisfaction is dead. Say Hello! to a new best practice

‍‍“Customer Success is just another buzzword, but does it really matter?”Buzzwords come and go. However,a quick peek into their journey helps assess which are likely to stay. Consider this global comparison of various Customer-related terms, in the past 1.5 decades (source: Google Trends). This is a simple analysis of using Google Trend’s ‘Interest over time’ feature.The search-terms ‘Customer Satisfaction’and ‘Customer Delight’ rapidly declined in the past 15 years. In other words –as concepts, these two terms have indeed faded.Instead, ‘Customer Success' and ‘Customer Service’ show an almost contrary pattern. Fair to conclude that these two terms have steadily generated interest, possibly even become synonymous with customer best practices. “Alright, so Customer Success is a new best practice. Care to explain what it is?”Customer Success is not just another department/ function in an organization. It is a company-wide commitment to achieve success, marshal teams together and work towards adding more value to customers. It cannot lie buried in mission statements, as merely 'Customer Satisfaction'.Instead, it must manifest in real, everyday actions across business functions; actions that aim to deliver customer delight… be it Sales, Product Development, Research & Innovation,Human Resources or Finance.In today’s Everything-as-a-Service world, Customer Success is a Top 5 business ...


Read More on Datafloq

Demonstrating Leadership or managing response in times of Crisis

In the present scenario, the Coronavirus crisis is unfolding over an arc of time with a beginning, middle and end. The actions of executives and their teams now, in the midst of this crisis, will significantly determine their destiny.For nearly two decades, it is observed that public and private-sector executives are in high-stakes, high-pressure situations. The crises are most often under-led and over managed. The best leaders navigate jagged waters deftly, energizing organizations, saving lives, and inspiring communities. However, it is discovered that many leaders fall into one or more of the following leadership traps:Taking a narrow viewThe human brain is programmed to narrow its focus in the face of a threat. It’s an evolutionary survival mechanism designed for self-protection. The human brain is hard-wired to hyper-focus on only the threat. However, leaders need to be able to pull back and see the whole field—what’s happening right in front of you and all around you. Take a broad view, including near-term and long-term challenges. The focused meta-leadership fosters well-directed management.Managing the crisisManaging a crisis can give your adrenaline a boost and be thrilling, followed by a crash. Instead, keep taking that long view, and anticipate what’s to come in the ...


Read More on Datafloq

How AI and Machine Learning Are Revolutionizing Software Development

Artificial Intelligence is the hot topic of the 21st century and it continues to amaze us as it demonstrates its impact on almost all industries. Software development is arguably one of the most impacted industries and every aspect of it is being benefited by AI and machine learning technologies. Starting from prediction of project timeline to software delivery estimation and use in bug management tools, code assistance, and testing, AI and ML help programs in various ways.

AI and ML have played a major role in the development and testing tools, especially bug management tools. But in the coming years, we’ll see more AI-powered algorithms used in cloud services and more AI tools getting packaged through subscription-based services.

AI’s Role in Software Programming

Precise Delivery Estimation

It isn’t difficult for an experienced mobile app developer to give a near-perfect estimation of a software’s delivery. However, some factors, such as excess of coding and accommodating client requests, can delay the project beyond the delivery schedule. In this respect, AI-based analytics is a very resourceful tool that helps development companies predict the precise delivery time and analyze data of a lot of similar projects.

AI can predict more precise project deadlines and timelines by using all types ...


Read More on Datafloq

Covid-19 has made plain need for joint research: DST secretary Ashutosh Sharma

There are 25 projects to study the pathology of virus, 800 startups are mapped for Covid-specific products, says Department of Science & Technology Secretary Ashutosh Sharma.

Intel buys Moovit transit app for $900M to help develop robotaxis

Moovit is popular for helping commuters or tourists find the best way to a destination by showing them bus and train routes, bike paths and car-pooling options.

Faster clearance for Govt drones for Covid-19 work

Fast-track approval not for operations such as drone package delivery or spraying disinfectants, which local authorities were doing during lockdown

Azure Databricks Security Best Practices

Azure Databricks is a Unified Data Analytics Platform that is a part of the Microsoft Azure Cloud. Built upon the foundations of Delta Lake, MLflow, Koalas and Apache Spark^TM, Azure Databricks is a first party PaaS on Microsoft Azure cloud that provides one-click setup, native integrations with other Azure cloud services, interactive workspace, and enterprise-grade security to power Data & AI use cases for small to large global customers. The platform enables true collaboration between different data personas in any enterprise, like Data Engineers, Data Scientists, Business Analysts and SecOps / Cloud Engineering.

In this article, we will share a list of cloud security features and capabilities that an enterprise data team could utilize to bake their Azure Databricks environment as per their governance policy.

Azure Databricks Security Best Practices

Security that Unblocks the True Potential of your Data Lake

Learn how Azure Databricks helps address the challenges that come with deploying, operating and securing a cloud-native data analytics platform at scale.

Bring Your Own VNET

What does the Azure Databricks platform architecture look like, and how you could set it up in your own enterprise-managed virtual network, in order to do necessary customizations as required by your network security team.

Trust But Verify with Azure Databricks

Get visibility into relevant platform activity in terms of who’s doing what and when, by configuring Azure Databricks Diagnostic Logs and other related audit logs in the Azure Cloud.

Securely Accessing Azure Data Sources from Azure Databricks

Understand the different ways of connecting Azure Databricks clusters in your private virtual network to your Azure Data Sources in a cloud-native secure manner.

Data Exfiltration Protection with Azure Databricks

Learn how to utilize cloud-native security constructs to create a battle-tested secure architecture for your Azure Databricks environment, that helps you prevent Data Exfiltration. Most relevant for organizations working with personally identifiable information (PII), protected health information (PHI) and other types of sensitive data.

Enable Customer-Managed Keys with Notebooks

Azure Databricks notebooks are stored in the scalable management layer powered by Microsoft, and are by default encrypted with a Microsoft-managed per-workspace key. You could also bring your own key to encrypt the notebooks.

Simplify Data Lake Access with Azure AD Credential Passthrough

Control who has access to what data by using seamless identity federation with Azure AD under the hood, and get cloud-native visibility into who is processing the data and when. Please feel free to refer to cloud-native access control for ADLS Gen 2 and how to configure it using Azure Storage Explorer. Such access management controls, including role-based access controls, are seamlessly utilized by Azure Databricks as outlined in the passthrough article.

Azure Databricks is HITRUST CSF Certified

Azure Databricks is HITRUST CSF Certified to meet the required level of security and risk controls to support the regulatory requirements of our customers. It is in addition to the HIPAA compliance that’s applicable through Microsoft Azure BAA.

What’s Next?

Attend the Azure Databricks Security Best Practices Webinar and bookmark this page, as we’ll keep it updated with the new security-related capabilities & controls. If you want to try out the mentioned features, get started by creating an Azure Databricks workspace in your managed VNET.

--

Try Databricks for free. Get started today.

The post Azure Databricks Security Best Practices appeared first on Databricks.

Intern Tips for a Virtual Databricks Internship

Winter 2020 Interns at our Spark Social S’mores Event

At Databricks, we host interns year-round, and we love sharing their experiences working on impactful projects that help data teams solve the world’s toughest challenges.

This summer’s intern program will be entirely virtual, with interns working on our Engineering team from all over the world. It’s a huge shift so we’re lucky to have our outgoing winter intern class share their thoughts on moving from working in our San Francisco office to a virtual experience, where our interns returned home to finish their internships remotely.

Read on to hear more about their Databricks experiences and how to get the most out of your virtual internship.

Brandon — Clusters Team

What has been your proudest accomplishment?
My team gives me work and lets me go off to work on it on my own, trusting that I’ll get it done. They rely on the work I do and trust that even as an intern I’ll be able to get it done.

How can other interns get the most out of their internship experience?
Read the Databricks Technical Blogs. I was able to get a sense of what types of projects were being asked of previous interns. Additionally, take time to hop on a call with your manager and/or mentor before you start to set up a positive relationship with them early on!

Sarth — ML Core Team

If you could describe your time at Databricks in 1 word, what would it be?
Learning: I’ve done eight internships previously, and I’ve learned more at Databricks so far than any of my previous ones. I couldn’t have even imagined getting to work on the projects I’m working on here — things like contributing open-source to Apache Spark™.

What has been your favorite memory or experience so far?
It’s actually been really fun during this abnormal COVID-19 time. My team used to have really fun social hours, and once we all started working from home, we just pivoted to virtual social hours and they’re great! We play games and get a chance to hang out with each other during these times.

Carl — Observability Team

What has been surprising about your experience so far?
Definitely the transparency across the company and visibility of engineering efforts across different teams and organizations.

How can other interns get the most out of their internship at Databricks?
Work to drive your own project by participating fully in team stand-ups and planning meetings. As my internship progressed, I made an effort to do more of this and I had good outcomes from it. The teams were really receptive to my feedback and input.

Melanie — Growth Team

What skills have you learned or improved?
So many things! I learned everything from creating a design doc and holding a design review to the workflows needed to ship a feature to production to utilizing tooling to be productive in my day-to-day.

How have you stayed connected with your team through the WFH period?
Our team has been doing weekly team hangout calls, as well as a #growth-random Slack channel where we have conversations about non-work related things — everything from the food we’re eating to tips for taking care of dogs!

Jon — Delta Pipelines Team

What has been surprising about your experience at Databricks?
Truly how much trust and confidence the team has in interns. Interns are given real, big projects to work on throughout their time; we’re given the freedom and flexibility to find solutions on our own or the support and assistance when needed.

What tips and tricks can you share about interning from home?
Work with your manager to find the best schedule for you. I have found time to make sure I get a chance to work out or cook throughout the day by splitting my days into two or three chunks!

Dhruv — Data Team

What has been your proudest accomplishment?
I’ve taken time to set up meetings with all sorts of people to learn more about Databricks. I started within engineering, setting up coffee chats to get more acquainted with other team members and parts of the product. Through this process, I’ve gotten insights about other parts of the company as well.

What has been your favorite memory or experience so far?
All the different board game nights. Board game nights are a chance to get to hang out with co-workers (including full-time engineers and other interns) and socialize. They’re now virtual but still provide fun opportunities!

Andrew — Cloud Team

What has mentorship been like for you?
From Day One, I had a mentor assigned to me. This mentor helped me onboard and had daily syncs to go through trivial things. My mentor also gave me career advice and pre-quarantine travel advice.

I found that my other team members were also more than happy to help out. They are willing to drop what they’re working on for a moment to help you out if you’re stuck on something.

If you could describe your time at Databricks in one word, what would it be?
Ownership: you really are the owner of your project. I had full control over the project and was able to change the direction slightly as it made sense. You’re really trusted to do that!

Shubhra — Workspace Team

What has mentorship been like for you?
Great! I have the freedom to make design choices, but when I make mistakes there’s support for me to learn from them. Everybody is super helpful and patient.

What has been surprising about your experience at Databricks?
The overall scale and scope of my project; I expected an “intern” project, however it’s much bigger than that. I get to work on some core functionality and code that was written a super long time ago. As an intern, it has been an amazing opportunity to be able to contribute to such important pieces of our product.

Scott — Dev Tools Team

What skills have you learned or improved?
This was my first infrastructure role so I really learned all about deployment systems and build systems — everything from github webhooks to kubernetes.

What tips and tricks can you share about interning from home?
Make sure to not stay inside all day! I take any opportunity to go outside — walking my dog several times a day, doing exercises, and stretching.

Interested in joining our next class of interns? Check out our Careers Page.

--

Try Databricks for free. Get started today.

The post Intern Tips for a Virtual Databricks Internship appeared first on Databricks.

How the Role of Test Managers Has Evolved in The Age of Agile

Development and testing teams all around the globe are adopting the Agile way of doing things. This leaves test managers an important question to answer: What does this change mean for us and how will it impact our roles? Managers need not worry because they are as important and valuable as ever. Their roles are now even tougher as the pressure to deliver high-quality products in less time has now increased. This will require them to put all their knowledge and experience in trying to manage development and testing teams to produce expected results.Let’s have a look at some of the changes in test managers’ roles.From Managing to Enabling TeamsManagers are now required to be extra sharp with the ability to make rapid decisions when it comes to selecting an appropriate mix of manual and automated approaches as well as relevant test case management software. Managers must be able to quickly gather and process risk-related data to make informed decisions to ensure efficiency in testing. From Testing to CoachingTeams of testers usually consist of highly-skilled QA experts. But it doesn’t mean that they don’t face challenges. They face their own set of challenges daily that need to be taken ...


Read More on Datafloq

Here’s How Digital Transformation is Altering the Security Needs of Businesses

Digital transformation has now changed how business is carried out. More number of companies are adopting advanced technologies such as Cloud, IoT, Big Data, Data Analytics, Machine Learning. Businesses today are looking towards Cloud environments for securing their business-critical data. These advanced technologies have forced businesses to ponder over their existing business model again. The businesses also need to devise new ways in which data could be utilized to the fullest for propelling their overall growth. Digital transformation does bring in a lot of value-added benefits. Still, on the other hand, it also has some challenges that businesses need to consider seriously. The top-most challenge remains the dynamic nature of security needs that are backed by the digital presence of any growing business. An online survey stated that nearly 85% of information security officials perceive security issues related to digital transformation to cause a significant amount of effect on their companies.So, in the following article, I shall be discussing what role does digital transformation play in altering the security needs of a business. Here are some changes that the businesses need to be extremely cautious about-1. Expanded Attack BaseMore the number of applications, data, ...


Read More on Datafloq

Security Challenges Businesses Face During Covid-19

The rise of COVID-19 has raised new concerns for organizations with respect to security. The rapid spread of this virus has caused many businesses to close down, however, digital businesses are still surviving due to their technological revolution. Since almost all countries are in a complete lockdown situation, businesses are trying to survive online, which makes it more challenging with respect to security. In this scenario, a cyber security testing company comes up with ways to combat these challenges and make through these difficult times with ease. Security Challenges in the Current ScenarioThis is going to be a huge transition, and due to the speed of this sudden change has also caught many organizations by surprise as in some cases there are deployments without proper governance and interoperability within the existing electronic medical records (EMRs). We know that cyber security is an important pillar in digital transformation. But due to this pandemic situation organizations have to be more careful and consider security above anything else. Reports from various countries have highlighted the importance of managing security even more than usual in a pandemic.This has not come as a surprise for those companies who already support remote working. However, it is ...


Read More on Datafloq

Cryptocurrency exchanges approach RBI seeking clarity on status and taxability

The crypto exchanges have written to RBI and sought clarity on their status as lenders continue to deny banking services to them due to a lack of clear instructions from the regulator.

C-CAMP to invest Rs 1 cr in startup doing testing kits

C-CAMP is helping CoSara Diagnostics, Huwel Life Sciences and DNAXperts complete regulatory processes and scale up capacities quickly.

AI Powered Contract Analysis and Review | Smart Contract Analytics

Smart Contract Analytics (SCA) Platform is based on AI / ML and specifically uses NLP and pattern recognition to process the contracts and extract the metadata. The platform is hosted on the AWS cloud that takes advantage of all security features that AWS provides along with the high availability and scalability features with GDPR compliances.Artificial Intelligence / Machine Learning capabilities:The SCA Platform uses Deep learning Neural Network models like Bi-LSTM, CNN, and other models that leverage state-of-the-art Natural Language Processing (NLP) based modeling techniques such as Word2Vec and Bi-directional Encoder Representations with Transformers to develop a semantic understanding of the document contents and further extract relevant entities and clauses. Smart Contract Analytics helps professionals complete contract review and analysis faster and more accurately than conventional methods. Artificial intelligence Powered Contract Analysis solution with machine learning and NLP changing the way contracts are analyzed and reviewed.


Product features:1) NLP and deep learning techniques for extracting metadata2) Export the extracted data to Excel or client desired format and push the extracted data into any CLM or downstream system3) Support of foreign languages (currently supported languages include Spanish and German)and rest of the ...


Read More on Datafloq

In Data-Driven Organisations, IT should be the Driver for Innovation

The world is changing faster than ever before, which was already the case before this terrible COVID-19 pandemic. The Coronavirus is a typical Black Swan that is changing the game forever. Although several people and organisation saw this pandemic coming and warned us extensively, we did not do enough to prevent it from happening. As a result, for the majority of the people and organisation, it is an unexpected event that has a tremendous impact, and in hindsight, we could have seen it coming.
Nevertheless, the Coronavirus pandemic is a catalyst for change. We can see that digital is becoming more critical than ever and organisations should welcome the digital employee. The digital employee is someone who works remotely – currently from home, but after the pandemic, this can be anywhere in the world – and uses various tools to do their work and be productive. The digital employee will offer a lot of benefits for the employer as well as for the employee, and it changes how organisations run their business.
Apart from the digital employee, the rise of emerging information technologies also changes how we organise activities. The combination of the pandemic and the continuous development of new, ever more ...


Read More on Datafloq