AI Machine Learning Efforts Encounter A Carbon Footprint Blemish

By Lance Eliot, the AI Trends Insider

Green AI is arising.

Recent news about the benefits of Machine Learning (ML) and Deep Learning (DL) has taken a slightly downbeat turn toward pointing out that there is a potential ecological cost associated with these systems. In particular, AI developers and AI researchers need to be mindful of the adverse and damaging carbon footprint that they are generating while crafting ML/DL capabilities.

It is a so-called “green” or environmental wake-up call for AI that is worth hearing.

Let’s first review the nature of carbon footprints (CFPs) that are already quite familiar to all of us, such as the carbon belching transportation industry.

A carbon footprint is usually expressed as the amount of carbon dioxide emissions spewed forth, including for example when you fly in a commercial plane from Los Angeles to New York, or when you drive your gasoline-powered car from Silicon Valley to Silicon Beach.

Carbon accounting is used to figure out how much a machine or system produces in terms of its carbon footprint when being utilized and can be calculated for planes, cars, washing machines, refrigerators, and just about anything that emits carbon fumes.

We all seem to now know that our cars are emitting various greenhouse gasses including the dreaded carbon dioxide vapors that have numerous adverse environmental impacts. Some are quick to point out that hybrid cars that use both gasoline and electrical power tend to have a lower carbon footprint than conventional cars, while Electrical Vehicles (EV’s) are essentially zero carbon emissions at the tailpipe.

Calculating Carbon Footprints For A Car

When ascertaining the carbon footprint of a machine or device, it is easy to fall into the mental trap of only considering the emissions that occur when the apparatus is in use. A gasoline car might emit 200 grams of carbon dioxide per kilometer traveled, while a hybrid-electric might produce about half at 92 grams, and an EV presumably at 0 grams, per EPA and Department of Energy.

See this U.S. government website for detailed estimates about carbon emissions of cars: https://www.fueleconomy.gov/feg/info.shtml#guzzler

Though the direct carbon footprint aspect does indeed involve what happens during the utilization effort of a machine or device, there is also the indirect carbon footprint that requires our equal attention, involving both upstream and downstream elements that contribute to a fuller picture of the true carbon footprint involved. For example, a conventional gasoline-powered car might generate perhaps 28 percent of its total life-time carbon dioxide emissions when the car was originally manufactured and shipped to being sold.

You might at first be normally thinking like this:

• Total CFP of a car = CFP while burning gasoline

But it should be more like this:

• Total CFP of a car = CFP when the car is made + CFP while burning gasoline

Let’s define “CFP Made” as a factor about the carbon footprint when a car is manufactured and shipped, and another factor we’ll call “CFP FuelUse” that represents the carbon footprint while the car is operating.

For the full lifecycle of a car, we need to add more factors into the equation.

There is a carbon footprint when the gasoline itself is being generated, I’ll call it “CFP FuelGen,” and thus we should include not just the CFP when the fuel is consumed but also when the fuel was originally processed or generated. Furthermore, once a car has seen its day and will be put aside and no longer used, there is a carbon footprint associated with disposing or scrapping of the car (“CFP Disposal”).

This also brings up a facet about EV’s. The attention of EV’s as having zero CFP at the tailpipe is somewhat misleading when considering the total lifecycle CFP since you should also be including the carbon footprint required to generate the electrical power that gets charged into the EV and then is consumed while the EV is driving around. We’ll assign that amount to the CFP FuelGen factor.

The expanded formula is:

• Total CFP of a car = CFP Made + CFP FuelUse + CFP FuelGen + CFP Disposal

Let’s rearrange the factors to group together the one-time carbon footprint amounts, which would be the CFP Made and CFP Disposal, and group together the ongoing usage carbon footprint amounts, which would be the CFP FuelUse and CFP FuelGen. This makes sense since the fuel used and the fuel generated factors are going to vary depending upon how much a particular car is being driven. Presumably, a low mileage driven car that mainly sits in your garage would have a smaller grand-total over its lifetime of the CFP consumption amount than would a car that’s being driven all the time and racking up tons of miles.

The rearranged overall formula is:

• Total CFP of a car = (CFP Made + CFP Disposal) + (CFP FuelUse + CFP FuelGen)

Next, I’d like to add a twist that very few are considering when it comes to the emergence of self-driving autonomous cars, namely the carbon footprint associated with the AI Machine Learning for driverless cars.

Let’s call that amount as “CFP ML” and add it to the equation.

• Total CFP of a car = (CFP Made + CFP Disposal) + (CFP FuelUse + CFP FuelGen) + CFP ML

You might be puzzled as to what this new factor consists of and why it is being included. Allow me to elaborate.

AI Machine Learning As A Carbon Footprint

In a recent study done at the University of Massachusetts, researchers examined several AI Machine Learning or Deep Learning systems that are being used for Natural Language Processing (NLP) and tried to estimate how much of a carbon footprint was expended in developing those NLP systems (see the study at this link here: https://arxiv.org/pdf/1906.02243.pdf).

You likely already know something about NLP if you’ve ever had a dialogue with Alexa or Siri. Those popular voice interactive systems are trained via a large-scale or deep Artificial Neural Network (ANN), a kind of computer-based model that simplistically mimics brain-like neurons and neural networks, and are a vital area of AI for having systems that can “learn” based on datasets provided to them.

Those of you versed in computers might be perplexed that the development of an AI Machine Learning system would somehow produce CFP since it is merely software running on computer hardware, and it is not a plane or a car.

Well, if you consider that there is electrical energy used to power the computer hardware, which is used to be able to run the software that then produces the ML model, you could then assert that the crafting of the AI Machine Learning system has caused some amount of CFP via however the electricity itself was generated to power the ML training operation.

According to the calculations done by the researchers, a somewhat minor or modest NLP ML model consumed an estimated 78,468 pounds of carbon dioxide emissions for its training, while a larger NLP ML consumed an estimated 626,155 pounds during training. As a basis for comparison, they report that an average car over its lifetime might consume 126,000 pounds of carbon dioxide emissions.

A key means of calculating the carbon dioxide produced was based on the EPA’s formula of total electrical power consumed is multiplied by a factor of 0.954 to arrive at the average CFP in pounds per kilowatt-hour and as based on assumptions of power generation plants in the United States.

Significance Of The CFP For Machine Learning

Why should you care about the CFP of the AI Machine Learning for an autonomous car?

Presumably, conventional cars don’t have to include the CFP ML factor since a conventional car does not encompass such a capability, therefore the factor would have a value of zero in the case of a conventional car. Meanwhile, for a driverless car, the CFP ML would have some determinable value and would need to be added into the total CFP calculation for driverless cars.

Essentially, it burdens the carbon footprint of a driverless car and tends to heighten the CFP in comparison to a conventional car.

For those of you that might react instantly to this aspect, I don’t think though that this means that the sky is falling and that we should somehow put the brakes on developing autonomous cars, you ought to consider these salient topics:

• If the AI ML is being deployed across a fleet of driverless cars, perhaps in the hundreds, thousands, or eventually millions of autonomous cars, and if the AI ML is the same instance for each of those driverless cars, the amount of CFP for the AI ML production is divided across all of those driverless cars and therefore likely a relatively small fractional addition of CFP on a per driverless car basis.
• Autonomous cars are more than likely to be EVs, partially due to the handy aspect that an EV is adept at storing electrical power, of which the driverless car sensors and computer processors slurp up and need profusely. Thus, the platform for the autonomous car is already going to be significantly cutting down on CFP due to using an EV.
• Ongoing algorithmic improvements in being able to produce AI ML is bound to make it more efficient to create such models and therefore either decrease the amount of time required to produce the models (accordingly likely reducing the electrical power consumed) or can better use the electrical power in terms of faster processing by the hardware or software.
• For semi-autonomous cars, you can expect that we’ll see AI ML being used there too, in addition to the fully autonomous cars, and therefore the reality will be that the CFP of the AI ML will apply to eventually all cars since conventional cars will gradually be usurped by semi-autonomous and fully autonomous cars.
• Some might argue that the CFP of the AI ML ought to be tossed into the CFP Made bucket, meaning that it is just another CFP component within the effort to manufacture the autonomous car. And, if so, based on preliminary analyses, it would seem like the CFP AI ML is rather inconsequential in comparison to the rest of the CFP for making and shipping a car.

For those of you interested in trying out an experimental impact tracker in your AI ML developments, there are various tools coming available, including for example this one posted at GitHub that was developed jointly by Stanford University, Facebook AI Research, and McGill University: https://github.com/Breakend/experiment-impact-tracker.

As they say, your mileage may vary in terms of using any of these emerging tracking tools and you should proceed mindfully and with appropriate due diligence for applicability and soundness.

For my framework about AI autonomous cars, see the link here: https://aitrends.com/ai-insider/framework-ai-self-driving-driverless-cars-big-picture/

Why this is a moonshot effort, see my explanation here: https://aitrends.com/ai-insider/self-driving-car-mother-ai-projects-moonshot/

For more about the levels as a type of Richter scale, see my discussion here: https://aitrends.com/ai-insider/richter-scale-levels-self-driving-cars/

For the argument about bifurcating the levels, see my explanation here: https://aitrends.com/ai-insider/reframing-ai-levels-for-self-driving-cars-bifurcation-of-autonomy/

Conclusion

There’s an additional consideration for the CFP of AI ML.

You could claim that there is a CFP AI ML for the originating of the Machine Learning model that will be driving the autonomous car, and then there is the ongoing updating and upgrading involved too.

Therefore, the CFP AI ML is more than just a one-time CFP, it is also part of the ongoing grouping too.

Let’s split it across the two groupings:

• Total CFP of a car = (CFP Made + CFP Disposal + CFP ML1) + (CFP FuelUse + CFP FuelGen + CFP ML2)

You can go even deeper and point out that some of the AI ML will be taking place in-the-cloud of the automaker or tech firm and then be pushed down into the driverless car (via Over-The-Air or OTA electronic communications), while some of the AI ML might be also occurring in the on-board systems of the autonomous car. In that case, there’s the CFP to be calculated for the cloud-based AI ML and then a different calculation to determine the CFP of the onboard AI ML.

There are some that point out that you can burden a lot of things in our society if you are going to be considering the amount of electrical power that they use, and perhaps it is unfair to suddenly bring up the CFP of AI ML, doing so in isolation of the myriad of other ways in which CFP arises due to any kind of computer-based system.

In the case of autonomous cars, it is also pertinent to consider not just the “costs” side of things, which includes the carbon footprint factor, but also the benefits side of things.

Even if there is some attributable amount of CFP for driverless cars, it would be prudent to consider what kinds of benefits we’ll derive as a society and weigh that against the CFP aspects. Without taking into account the hoped-for benefits, including the potential of human lives saved, the potential for mobility access to all and including the mobility marginalized, and other societal transformations, you get a much more robust picture.

In that sense, we need to figure out this equation:

• Societal ROI of autonomous cars = Societal benefits – Societal costs

We don’t yet know how it is going to pan out, but most are hoping that the societal benefits will readily outweigh the societal costs, and therefore the ROI for self-driving driverless autonomous cars will be hefty and leave us all nearly breathless as such.

Copyright 2020 Dr. Lance Eliot

This content is originally posted on AI Trends.

[Ed. Note: For reader’s interested in Dr. Eliot’s ongoing business analyses about the advent of self-driving cars, see his online Forbes column: https://forbes.com/sites/lanceeliot/]

AI Helping to Transform Education in Pandemic Era

By AI Trends Staff

The impact of the COVID-19 pandemic on education has been profound, with new ways of thinking about how best to teach students reverberating in institutions of higher learning, K-12 classrooms and in the business community.

The role of AI is central to the discussion on every level. For the K-12 classroom, teachers are thinking about how to use AI as a teaching tool. For example, Deb Norton of the Oshkosh Area school district in Wisconsin, was asked several years ago by the International Society for Technology in Education to lead a course on the uses of AI in K-12 classrooms, according to a recent account in Education Week.

The course includes sections on the definition of artificial intelligence, machine learning, voice recognition, chatbots and the role of data in AI systems. To teach about machine learning, one teacher tied it to yoga, and how the student could do a yoga pose that could be recognized via machine learning, and then the machine could give them feedback on their yoga poses.

Another teacher working with elementary students used the coding site Scratch to create interactive characters and programs such as for creating a skill in Amazon’s Alexis, which are like apps on a smart phone except activated with voice.

Deb Norton, teacher, Oshkosh Area school district, Wisconsin

Asked if she foresees increasing interest in AI as a result of increased remote learning during the pandemic, Norton stated, “AI could become a really big part of virtual learning and at-home learning, but I just don’t think we’re quite there yet. For many of our educators, they’re just dipping their feet into how this would work.”

Protecting privacy is an issue. Many schools will not allow schools to open up Alexa and Google Home out of concern for personal privacy. One workaround could be a school-only network to serve as a test bed.

She does see the potential for AI to help with learning management applications “from a teacher-educator point of view, to be able to engage and monitor and track the types of lessons and strategies that can be delivered in the most effective way in the classroom.”

Investor Sees Disruption Ahead in Higher Education

From an investment point of view, AI in education in the new era represents opportunity. Some see disruption looming in the higher education university system as a result.

“A reckoning is coming for schools and universities,” stated Scott Galloway,  a professor of marketing at the NYU Stern School of Business, in a recent account in TechCrunch.  “We’ve raised prices 1400% but if you walked into a classroom today it wouldn’t look, smell or feel much different from what it did 40 years ago.”

Likening it to a shrinkage in retail – which saw 9,500 closures in 2019 and more than 15,000 so far in 2020 – he predicts a sustained drop in applications for four-year universities, with dozens if not hundreds of colleges and universities unable to recover.

Scott Galloway, professor of marketing, NYU Stern School of Business

Roei Deutsch, co-founder and CEO of live video course marketplace Jolt Inc., stated during a talk on the Coffee Break podcast, “The blow to the world of higher education was bound to come. There is a higher education bubble, something there does not work in terms of cost versus what students receive in return, and you can say that the coronavirus crisis is the beginning of this bubble’s bursting.”

Thus the virus is seen as accelerating a trend that was already underway. The global corporate e-learning market is estimated to grow up to $30 billion at a 13% compound annual growth rate through 2022. “This growth was driven in large part by the increased importance of matching workforce capabilities with actual required skill sets,” stated Joe Apprendi, a general partner at Revel, a venture capital firm formed by business founders, author of the TechCrunch article.

New core education products, as suggested by teacher Norton,  include learning experience platforms (LXP) and learning management systems (LMS), used to monitor, track and administer employment learning activities.

Learning software is primarily designed to create more personalized learning experiences and help users discover new learning opportunities by combining learning content from different sources, while recommending and delivering them — with the support of AI — across multiple digital touch points such as desktop applications and mobile learning apps.

Colleges, universities and enterprises are all looking at these tools. Instead of building training academics to help train people for new or expanded roles in an organization, “Enterprises will now target the front end of the recruiting funnel where higher education begins,” Apprendi suggests. “The potential for global enterprises to own the university experience is suddenly, very real.” The online faculty could be professors from shuttered universities. A hybrid, for-profit model that blends universities and global enterprises could emerge, along the lines of the US Naval Academy, where a tuition-free education comes with an obligation to serve for a period of time.

“Students could see debt cut in half and have a clear path forward toward employment,” he stated. Whatever landscape emerges, changes are in store for universities and colleges.

Software Helping with Remote Learning Challenges

Meanwhile back in K-12 education, the transition to remote learning has been challenging. Many students fail to log into classrooms or complete assignments, according to a recent account in TechRepublic. The number of students logging in has declined by 43% since the start of school closures, and the number of students completing at least one virtual lesson has dropped by 44%, according to a report from Achieve3000. The report was based on data from 1.6 million students across 1,364 school districts.

The transition to e-learning is particularly difficult for struggling readers, who need more time and individual assistance with lessons, the report found. An innovative approach is taken by AI-powered software Amira, designed to remotely help students become better readers.

Amira has been recognized, such as with a nomination for a Codie Award for Best Use of Emerging Technology for Learning in Education. Amira is an intelligent reading assistant designed from decades of research on the science of reading from the University of Texas, and AI in support of reading development from Carnegie Mellon University.

“Amira listens, delivers in-the-moment error-specific feedback, and reports progress for every reading session,” stated Sara Erickson, Amira’s vice president of customer success. “Amira is changing how teachers focus their reading instruction with the help of machine learning to accelerate student reading growth.”

As the student reads, Amira uses AI to decipher what obstacles the young reader is facing, delivering micro-interventions that help to bridge the reading skills gaps. The software helps assess reading fluency, pinpoint errors, and help improve those weaknesses.

“Teachers will never be replaced by software, but they can be supported by it,” Erickson stated. Approximately 125 school districts are currently using Amira, with the K-3 student population in those districts totaling more than 600,000.

Read the source articles in Education Week, TechCrunch and TechRepublic.

AI-Based Tools Predict COVID-19 Disease Severity

By Paul Nicolaus, Science Writer

Two healthcare workers under the age of 30 fell ill in Wuhan, China, where the first COVID-19 case was reported. One survived. The other wasn’t as fortunate. But why?

It’s an example researchers at the Radiological Society of North America highlighted while pointing out that this phenomenon—some patients falling critically ill and dying as others experience minimal symptoms or none at all—is one of the most mysterious elements of this disease. Mortality does correlate with factors such as age, gender, and some chronic conditions. Considering young and previously healthy individuals have succumbed to this virus, though, there could be more complex prognostic factors involved.

Current diagnostic tests determine whether or not individuals have the virus. They do not, however, offer clues as to just how sick a COVID-positive patient could become. For the time being, clinicians cannot easily predict which patients who test positive will require hospital admission for oxygen and possible ventilation.

Because most cases are mild, identifying those at risk for severe and critical cases early on could help healthcare facilities prioritize care and resources such as ventilators and ICU beds. Figuring out who is at low risk for complications could be useful, too, as this could reduce hospital admissions while these patients are managed at home. As health systems across the globe continue to deal with large numbers of COVID-19 cases, new and emerging technologies may be able to help in this regard.

AI Plus Imaging

Researchers have been probing he use of AI and imaging to determine who has COVID-19, but some groups are taking a different approach and using this same combination to determine which patients are most likely to need the most extensive treatment.

In a paper published July 22 in Radiology: Artificial Intelligence (doi: 10.1148/ryai.2020200079), researchers at Massachusetts General Hospital and Harvard Medical School reveal efforts to develop an automated measure of COVID-19 pulmonary disease severity using chest radiographs (CXRs) and a deep-learning algorithm.

Elsewhere, an international group proposed an AI model that uses COVID-19 patients’ geographical, travel, health, and demographic data to predict disease severity and outcome. Future work is expected to focus on the development of a pipeline that combines CXR scanning models with these types of healthcare data and demographic processing models, according to their paper published July 3 in Frontiers in Public Health (doi: 10.3389/fpubh.2020.00357).

In June, GE Healthcare announced a partnership with the University of Oxford-led National Consortium of Intelligent Medical Imaging (NCIMI) in the UK to develop algorithms aimed at predicting COVID-19 severity, complications, and long-term impact.

Similarly, experts at the University of Copenhagen set out to create models that calculate the risk of a COVID-19 patient’s need for intensive care. The algorithms are designed to find patterns among Danish coronavirus patients who have been through the system to find shared traits among the most severely affected. The patterns are compared with data gathered from recently hospitalized patients, such as X-rays, and sent to a supercomputer to predict how likely a patient is to require a ventilator and how many days will pass before that need arises.

Meanwhile, researchers at Case Western Reserve University are using computers to find details in digital images of chest scans that are not easily seen by the human eye to quickly determine which patients are most likely to experience further deterioration of their health and require the use of ventilators.

“The approach we’ve taken is actually to create a synergistic artificial intelligence algorithm—one that combines patterns from CT scans with clinical parameters based on lab values,” Anant Madabhushi, professor of biomedical engineering at Case Western Reserve and head of the Center for Computational Imaging and Personalized Diagnostics (CCIPD) told Diagnostics World.

Anant Madabhushi, professor of biomedical engineering, Case Western Reserve University

“And the secret sauce, if you will, is the fact that we’re using neural networks and deep learning to automatically go into the CT scans and identify exactly where the region of disease is,” he added. Zeroing in on the disease presentation on the CT scan makes it possible to mine patterns using the neural networks from those regions and combine them with the clinical parameters.

Madabhushi and colleagues have completed a multi-site study that included nearly 900 patients from Wuhan, China, and Cleveland, Ohio. They found that the combination of the clinical parameters and imaging features yielded a higher predictive accuracy in identifying who would go on to need a ventilator compared to a model that uses the imaging features alone and also compared to a model that used only the clinical parameters.

The inspiration for this work came about months ago as Italy hit its peak and the country’s hospitals were overwhelmed with patients who couldn’t breathe. Some of the stories were gut-wrenching, he explained, particularly the ones that highlighted how physicians had to make case by case determinations about who got a ventilator and who didn’t.

“It really got me thinking about what the implications are for the US or the rest of the world,” he said, if a second wave materializes in the fall as some experts have predicted. Of course, we are not out of the first wave yet, he acknowledged, but there is a real concern that a second wave could be even deadlier than the first considering it would take place during flu season.

Madabhushi and colleagues began building their model using images and datasets found online in early March. In April, the CCIPD was offered digital images of chest scans taken from roughly 100 early victims of the novel coronavirus from Wuhan, China. Using that information, the researchers developed machine learning models to predict the risk of a COVID-19 patient needing a ventilator—one based on neural networks and another derived from radiomics.

Early CT scans from patients with COVID-19 showed distinctive patterns specific to those in the intensive care unit (ICU) compared to those not in the ICU. Initially, the research team was able to achieve an accuracy of roughly 70% to 75%. Since then, they have improved upon that performance metric, he said, raising the accuracy level to about 84%.

They have worked to circumvent bias by exposing the AI to patients from different demographics, ethnicities, populations, and scanners. But there’s still work to be done, including additional multi-site testing and prospective field testing. Madabhushi hopes to validate the technology on patients from the Louis Stokes Cleveland VA Medical Center, where he is a research scientist, and is looking to prove the technology at Cleveland Clinic as well.

The team is also developing a user interface that couples the AI with a tool that allows the end-user to enter a CT scan and clinical parameters to see the likelihood of needing a ventilator. Before clinically deploying the technology, he wants to put this in the hands of end-users for additional prospective field testing so that users can get comfortable with the tool, get a sense of how to work with it, and learn how to interpret and use the results coming out of it.

Rather than making arbitrary decisions about who gets a ventilator and who does not, the big hope is that this type of triaging technology could enable more rational decision-making for appropriating resources.

AI and Blood Biomarkers

Another group was also motivated by the scenario that played out in northern Italy back in February and March as a lack of ICU beds led to tough decisions for clinicians.

“Unfortunately, this process, I would say, is a little bit cyclical,” John T. McDevitt, professor of biomaterials at NYU College of Dentistry and professor of chemical and molecular engineering at NYU Tandon School of Engineering told Diagnostics World. Similar scenarios have played out in New York City, for instance, and more recently in Houston. “When you hit this point where you don’t have any buffer, any excess capacity, then it forces a very difficult situation.”

John T. McDevitt, professor of chemical and molecular engineering at NYU Tandon School of Engineering

He wants to provide clinicians with what he describes as “a flashlight that goes into this dark room of COVID-19 severity.” The intent is to look into the future and attempt to figure out which patients will perish unless extreme measures are taken, which patients should be admitted to the hospital, and which patients can safely recover from home.

“I would describe this as the third leg of the stool for the diagnosis and prognosis of COVID-19,” he explained. PCR testing has been used to determine whether individuals have the disease, and serology testing has helped establish whether people have had the condition in the past. The missing leg here, he said, has been determining which patients are going to end up in the hospital and which patients are most likely to perish.

To fill that void, he and colleagues have developed a smartphone app that uses AI and biomarkers in patients’ blood to determine COVID-19 disease severity. Their findings were published June 3 in Lab on a Chip (doi: 10.1039/D0LC00373E).

Relying on data from 160 hospitalized COVID-19 patients in Wuhan, China, they found four biomarkers measured in blood tests that were elevated in the patients who died compared with those who recovered. These biomarkers (C-reactive protein, myoglobin, procalcitonin, and cardiac troponin I) can signal complications relevant to COVID-19, such as reduced cardiovascular health, acute inflammation, or lower respiratory tract infection.

The researchers then developed a model using the biomarkers as well as age and sex—two risk factors. They trained the model to define the patterns of COVID-19 disease and predict its severity. When a patient’s information is entered, the model comes up with a numerical severity score ranging from 0 (mild) to 100 (critical), reflecting the probability of death from the complications of COVID-19.

It was validated using information from 12 hospitalized COVID-19 patients from Shenzhen, China, and further validated using data from over 1,000 New York City patients. The app has also been evaluated in the Family Health Centers at NYU Langone in Brooklyn.

The diagnostic system uses small samples, such as swabs of saliva or drops of blood from a fingertip, which are added to credit card-sized cartridges. The cartridge is put into a portable analyzer that tests for a range of biomarkers, with results available in under 30 minutes. After optimizing the app’s clinical utility, the goal is to roll it out nationwide and worldwide.

Over the coming months, McDevitt’s laboratory, in partnership with SensoDx—a company spun out of his lab—intends to develop and scale the ability to produce a severity score similar to the way people with diabetes check their blood sugar. The plan is to distribute the tool first to disease epicenters to maximize its impact considering not all locations are dealing with a shortage of ICU beds or respirators.

McDevitt also highlighted the potential to help address racial disparities. “COVID has ripped the scab off of this particular wound,” he said. This technology can help level the healthcare playing field and remove some of the unintentional racial or ethnic biases that may weave their way into the delivery of healthcare. By putting the severity score on a numerical index, it arguably provides a more objective way to make challenging pandemic-related healthcare decisions.

McDevitt and colleagues aren’t the only ones pursuing blood-based biomarkers for the prediction of COVID-19 disease severity.

Another example can be found in a study published May 14 in Nature Machine Intelligence (doi: 10.1038/s42256-020-0180-7) and conducted by a group of Chinese researchers, who used a database of blood samples from nearly 500 infected patients in the Wuhan region.

Their machine learning-based model predicts the mortality rates of patients over 10 days in advance with more than 90% accuracy, according to the paper, using three biomarkers: lactic dehydrogenase, lymphocyte, and high-sensitivity C-reactive protein.

Paul Nicolaus is a freelance writer specializing in science, nature, and health. Learn more at www.nicolauswriting.com. This article was originally published in Diagnostics World.

Startup: Andrea Roberson of AI4US Seeks Diversity in AI, Tech Workforce

By AI Trends Staff

Andrea Roberson is CEO and founder of AI4US, which aims to address the lack of racial and gender diversity in high technology. She was previously a Machine Learning Researcher in the Economic Statistical Methods Division (ESMD) of the US Census Bureau, where she worked for over a decade. She is a graduate of Stony Brook College, where she is the first Black woman to earn a PhD in Applied Math and Statistics,  and of Spelman College, where she earned a degree in mathematics.

Andrea Roberson, Founder and CEO, VenusAI4US

The company in the fall of 2020 started a program called RAPIDS, which places students in virtual cohorts of 20 to 25 students to take Fundamentals of Accelerated Data Science from Nvidia’s Deep Learning Institute, as described in a recent account in VentureBeat. The course is given as a three semester-credit course and is the first to be taught by Black women instructors.

AI Trends asked Roberson to respond to queries we ask of selected startups. Here are her responses received via email:

Describe your team, the key people.

Outside of our Founder, Andrea Roberson, our team mostly consists of volunteers. We are pulling together Black Women who are professors and industry leaders in AI.

What business problem are you trying to solve?

Our program is trying to address some of the inequities in post secondary education, specifically for Women of Color (WOC). A lot of recent research, even post-Covid, shows that a new workforce education system & new workforce training models can lead to meaningful change. Our program is seeking to make both strong learning and career outcomes. It is a hybrid of all the new post secondary models that have started to spring up in the last few years.

Higher Ed has not been educating for the workplace; Higher Ed has largely focused on foundational skills. The pandemic is the ideal time to do things differently, give the workforce more agile options. The prohibitive costs of Higher Ed (especially in the age of Covid)  have impeded access to WOC.

The Higher Ed outcome would be to reverse the trend of declining computing degree recipients.  Initially it would be offering a Master’s degree alternative, showing that we can be more effective than Spelman, Howard, and other Historically Black Colleges and University’s (HBCUs). Demonstrating that we can outperform than the entire Higher Ed system. The mean number of Master’s Degree recipients in computing is 72 for Black women between 2007-16. Overall, the number of degree recipients has dropped by 40% over the past decade, to 4%, from 7%.

Spelman College produces about the same amount of Math graduates in 2020 as it did in 1999, but the price has tripled. Theoretical algebra is not required for a lot of available data science positions, but it is required for Spelman’s credential. And unfortunately, I think most people in the Black community hold up the Spelman pathway through Higher Ed as the best pathway and perhaps one of the few pathways. I think Covid will upend those assumptions (even my own thinking). The limitations of Spelman’s infrastructure will be painfully and clearly revealed in the coming months and years. I believe we are on the precipice of seeing those computing degree completion rates diminish even further.

VenusAI4US aims to address the racial and gender diversity in AI and high technology by training women of color in needed skills. (Image from VenusAI4US)

How does your solution address the problem?

Within two years, we plan to produce more Black Women with the equivalent to a master’s degree in Data Science/Machine Learning & AI, than the entire US Higher Ed system. And double the participation of Black women in the AI workforce at an equitable cost per student.  Our curriculum is currently tailored to teaching for Black Lives, specifically WOC.  That means affirming Black identity and the beauty of Blackness in the classroom. As self-care for Black students and to directly confront anti-Blackness. The ferocity of racism in the United States against Black minds and Bodies, it just demands a different pedagogy.

AI4US will also mitigate the costs of Higher Ed by alternatives to standard financing approaches. E.g. Income share agreements and an outsourcing approach that sees students do contracted project-based work as part of training.

How are you getting to the market? Is there competition?

We haven’t found a program that specifically looks to improve the numbers of WOC receiving computing degrees and participating in the AI workforce.

Do you have any users or customers?

NVIDIA was our first customer, we have helped them increase the number of WOC that receive training from their Deep Learning Institute.

Any anecdotes/stories?

We believe building confidence is an essential cornerstone of building a career in AI. Research shows, among those students who get Bs or below, men become more likely than women to advance. Researchers concluded, “We need to find a way to message that a lot of people find this hard, and you can still excel in computing with Bs.”  We make it clear that working hard in these subjects does not signal a lack of talent. American Association of University Women found that “women experts portrayed as ‘superstars’ who are unique and exceptional have little impact—and sometimes have a deflating effect—on young women’s views of themselves.”

I received plenty of Bs in college and graduate school; I do not have a reputation as a math prodigy. But I still became the first Black woman to receive the PhD in Applied Math and Statistics from Stony Brook University. AI4US is creating an automated world, and we are determined that the future be filled with other women who could fathom making this choice too.

How is the company funded?

The initial years will be financed by corporate partners.

Learn more at VenusAI4US.

How Transition to Remote Work is Being Supported by AI

By John P. Desmond, AI Trends Editor

AI is supporting the transition to remote work, a transition which is now pointing towards fundamental changes such as disconnecting the company’s location from the worker’s location in hiring decisions.

Most companies have always preferred to hire locally. Job opportunities in Silicon Valley and New York City have made them centers of technology innovation, attracting top software engineering talent that have enabled the tech giants such as Google and Facebook to grow rapidly.

“But, with the creation of the new at-home economy, the local hiring structure is undergoing a once-in-a-lifetime transition,” stated Sanjit Singh Dang, PhD, a venture capitalist and co-founder and chairman of U First Capital, in a recent account in Forbes.

Sanjit Singh Dang, PhD, venture capitalist, Co-founder and Chairman, U First Capital

Companies are framing new remote work policies. Twitter announced that its employees can work from home indefinitely. Google announced in late July that its employees can work from home until July 2021. Facebook has said some 50% of its 45,000 employees could be working remotely long term, but the company said it plans to reduce pay for workers in locations with a lower cost of living.

Some 90% of Morgan Stanley’s 80,000 employees are working from home. Accenture CEO Julie Sweet has said that remote working is here to stay.

“Among all industries, Information Technology and AI have the maximum remote work feasibility,” Dr. Dang stated. “In fact, the adoption of AI may increase due to remote work capability.” Without a location barrier, recruiters will be able to hire talented AI professionals who may have been unwilling to relocate.” He calls this “the decentralization of AI talent” outside its university and tech hub centers roots.

Managing Remote Works is a Leadership Challenge

Now that we have all these remote workers, how do we manage them? Leadership scientist Dr. Tommy Weir founded enaible in Boston in 2018, on the idea that AI can help drive employee productivity. A survey of executives conducted by the company showed only four percent were measuring the productivity or remote workers, and 95% believed AI will play a role in doing so.

Dr. Tommy Weir, Founder and CEO, enaible, inc.

This transition to remote work was not planned; it was a result of COVID-19-related lockdowns. As a result, “It’s displaced work. Remote workers have home offices/workspaces set up, work designed for remote work, and normalcy. In this case, workers were told to pack their bags and work from home,” Dr. Weir stated in a recent interview with siliconAngle.

As a result, workers were not prepared and neither were their managers. Now an adjustment is going on. “This will require companies to rapidly emphasize using tools to aid remote management,” he stated. “The managers need prioritized, personalized recommendations on how to help their employees succeed.”

Software from enaible uses a Score to measure productivity. A Trigger-Task-Time algorithm helps managers understand how the workforce is performing. The Score also produces recommendations to managers, to help employees reach their performance goals.

“We micro-monitor without micromanaging,” Weir stated. “You need to have an eye on what people are doing and delivering. This is the perfect time to use an AI system that can do this better than humans. Allow the machines to learn and the humans to lead.”

Weir emphasized the importance of communication, and not just in Zoom calls. He suggested managers call employees on the phone on a regular basis.

How To Keep Your Head Together Working Remotely

Now that you are set up to work remotely and an AI program is helping you stay productive, how do you keep your head together?

A survey of 1,500 people, equal numbers of freelancers and office workers, found that freelancers were 86% more likely to self-report depression than officer workers, The survey was conducted by office supply company Viking and was reported in an account in Ozy.

More than six in 10 freelancers said their work made them lonely, and they were far more likely to report stress due to work than their officer-based counterparts.

Tom Miller is the CEO of ClearForce of Vienna, Va., a company that specializes in identifying employee stress and high-risk behavior. He suggests that business owners play an important role in ensuring an employee’s well-being, especially during the pandemic. “Executives, managers and team leaders need to take concrete steps to address employee well-being while also protecting the organization from new forms of insider risk,” he stated.

According to Miller, even outside the context of coronavirus and depression, organizations have to take responsibility for keeping tabs on employees to determine whether they’ve become disengaged or overly stressed.

While good mental health practices look different for different people, companies and co-workers need to be looking out for remote workers and try to stay connected despite the distance.

Read the source articles in Forbes, siliconAngle and Ozy.

Future of Work is Now: Intelligent Virtual Assistants Impact on the Customer Revenue Lifecycle

Scheduled for September 16, 2020, 1 pm to 2:00 pm EDT

REGISTER

While every business has its own objectives, the overarching goals remain the same: accelerate revenue while driving greater efficiency and cost optimizations. But in today’s economic climate with workforce and budget reductions that might be a tough mountain to climb. How can organizations adopt an augmented workforce using Intelligent Virtual Assistants to attract, acquire and grow customers at scale?

Come to this session to learn:

• How Intelligent Automation has migrated from the back-office to the front-office
• The challenges revenue-obsessed teams face in the wake of COVID-19 and how to overcome them
• Real examples of how IVAs have increased pipeline and revenue for enterprise organizations

Speakers:

David Schubmehl, Research Director, Cognitive/Artificial Intelligence Systems
Dave Schubmehl is Research Director for IDC’s Cognitive/Artificial Intelligent Systems and Content Analytics research. His research covers information access and artificial intelligence technologies including content analytics, search systems, unstructured information representation, cognitive computing, deep learning, machine learning, unified access to structured and unstructured information, Big Data, visualization, and rich media search in SaaS, cloud and installed software environments.  This research analyzes the trends and dynamics of the content analytics, discovery and cognitive systems software markets and the costs, benefits and workflow impacts of solutions that use these technologies.

Rashmi Vittal, Chief Marketing Officer
Rashmi brings to Conversica extensive experience in building marketing strategies and teams for both start-ups and large enterprises. Prior to joining Conversica, Rashmi led marketing for SAP Customer Data Cloud after the successful acquisition of Gigya, a customer identity management start-up, where she was responsible for the go-to-market strategy, product marketing, digital, content, communications and field marketing strategy. Rashmi has held various marketing leadership positions at IBM, Oracle and Neustar. Rashmi holds a Masters of Business Administration from the F.W. Olin Graduate School of Business at Babson College.

REGISTER

How is 5G Changing the Face of IoT?

Gone are the days when you had to spend a considerable amount of money to get your Internet data pack top-up. While a large number of people headed to the cyber cafes for accessing the Internet and doing any activity, it was a cumbersome task. Apart from this, video calls or accessing the Internet on the mobile freely were more than a distant affair.

However, thanks to the favorable policies around the world, the Internet has become cheaper than ever. Even if you look at the rural areas, you can easily find Internet services there. A lot of people today have smartphones or Internet connections in their homes that are seamless and provide good speed.

IoT as We Know Today

The reason why we’re highlighting the importance of the Internet is that it has become one of the most prized assets of the modern age. Be it our enterprises, individual activities, or a large part of business; everything heavily relies on the Internet. In fact, the world, as we know, today seems unimaginable without the powerful Internet.

The extent of popularization of services on the Internet has reached our households. In other words, we now have full-fledged appliances in our homes that are powered ...


Read More on Datafloq

Can 4G LTE Replace Home Internet? A Full Guide to Using LTE Modem

Some forecasts say that in 2020 more than 5 billion people will become members of the mobile world. At the same time, half of the entire population of the planet will have constant access to LTE network services. Further development progress will be linked to this innovative solution.LTE Network CharacteristicsBefore LTE, cellular communications could only operate at a certain speed rate. This created uncomfortable restrictions and, in general, slowed down the mobile communications processes. For example, 2G networks use only a few bands (850 MHz, 900 MHz, 1800 MHz, 1900 MHz). The same situation is in 3G, but with extra bands - 1900 and 2200 MHz. The progress of cellular communications was rapid and was divided into several generations - 1G, 2G, 3G. Still, the real leap in Internet connections took place only after the emergence of LTE, which supports data shipment up to 1Gbps. Unlike previous models, LTE can run on any frequency, both at the lowest and highest (450 MHz to 5 GHz). However, the main feature of this solution is the ability to unite numerous signals into one flow.The LTE itself lies in purely technical terms (the scheme and method of signals transmitted between phones and ...


Read More on Datafloq

How RPA is Expected to Affect the Future of Software Testing

Hyperautomation is a popular concept in the software testing industry. Similar to the Industrial Revolution, hyperautomation is a permanent transition to improve their manufacturing processes with the help of technology. However, even if organizations have still not made their way yet, robotic process automation (RPA) is expected to play an important role here.

The main reason why RPA is gaining popularity is its ability to help organizations with their legacy systems being more competitive by automating their workflow. Workflows here, mean repetitive activities that require low cognitive efforts, including those that are performed during software testing. RPA is a powerful tool, that replaces regression, performance, and load testing and QA experts can channelize their efforts on other testing activities as exploratory, usability, and Adhoc testing. An automation testing company explores all possible solutions to best utilize RPA in achieving its test automation efforts.

To explain RPA in simple words, it is an automation tool that automates a task by watching the user perform it in the app’s graphical user interface (GUI). This is the reason RPA is an appealing concept, and it enables codeless testing. With the unique features of RPA tools, testers can save their resources on a number of time-consuming ...


Read More on Datafloq

Jenkins graduates in the Continuous Delivery Foundation

We are happy to announce that the Jenkins project has achieved the graduated status in the Continuous Delivery Foundation (CDF). This status is officially effective Aug 03, 2020. Jenkins is the first project to graduate in the CD Foundation. Thanks to all contributors who made our graduation possible!

In this article, we will discuss what the CD Foundation membership and graduation mean to the Jenkins community. We will also talk about what changed in Jenkins as a part of the graduation, and what are the future steps for the project.

To know more about the Jenkins graduation, see also the announcement on the CD Foundation website. The press release is available here.

How does CDF membership help us?

About 18 months ago, Jenkins became one of the CDF founding projects, along with Jenkins X, Spinnaker and Tekton. A new foundation was formed to provide a vendor-neutral home for open source projects used for Continuous Delivery and Continuous Integration. Special interest groups were started to foster collaboration between projects and end user companies, most notably: Interoperability, MLOps and Security SIGs. Also, a Community Ambassador role was created to organize local meetups and to provide public-facing community representatives. Many former Jenkins Ambassadors and other contributors are now CDF Ambassadors, and they promote Jenkins and other projects there.

Thanks to this membership we addressed key project infrastructure needs. Starting from Jan 2020, CDF covers a significant part of the infrastructure costs including our services and CI/CD instances running on Microsoft Azure. The CD Foundation provided us with legal assistance required to get code signing keys for the Jenkins project. Thanks to that, we were able to switch to a new Jenkins Release Infrastructure. The foundation sponsors the Zoom account we use for Jenkins Online Meetups and community meetings. In the future we will continue to review ways of reducing maintenance overhead by switching some of our self-hosted services to equivalents provided by the Linux Foundation to CDF members.

Another important CDF membership benefit is community outreach and marketing. It helped us to establish connections with other CI/CD projects and end user companies. Through the foundation we have access to the DevStats service that provides community contribution statistics and helps us track trends and discover areas for improvement. On the marketing side, the foundation organizes webinars, podcasts and newsletters. Jenkins is regularly represented there. The CD Foundation also runs the meetup.com professional account which is used by local Jenkins communities for CI/CD and Jenkins Area Meetups. Last but not least, the Jenkins community is also represented at virtual conferences where CDF has a booth. All of that helps to grow Jenkins visibility and to highlight new features and initiatives in the project.

Why did we graduate?

The Jenkins project has a long history of open governance which is a key part of today’s project success. Starting from 2011, the project has introduced the governance meeting which are open to anyone. Most of the discussions and decision making happen publicly in the mailing lists. In 2015 we introduced teams, sub-projects and officer roles. In 2017 we introduced the Jenkins Enhancement Proposal process which helped us to make the key architecture and governance decisions more open and transparent to the community and the Jenkins users. In 2018 we introduced special interest groups that focus on community needs. In 2019 we have expanded the Jenkins governance board so that it got more bandwidth to facilitate initiatives in the project.

Since the Jenkins project inception 15 years ago, it has been steadily growing. Now it has millions of users and thousands of contributors. In 2019 it has seen 5,433 contributors from 111 countries and 272 companies, 67 core and 2,654 plugin releases, 45,484 commits, 7,000+ pull requests. In 2020 Q2 the project has seen 21% growth in pull requests numbers compared to 2019 Q2, bots excluded.

One may say that the Jenkins project already has everything needed to succeed. It is a result of continuous work by many community members, and this work will never end as long as the project remains active. Like in any other industry, the CI/CD ecosystem changes every day and sets new expectations from the automation tools in this domain. Just as the tools evolve, open source communities need to evolve so that they can address expectations, and onboard more users and contributors. The CDF graduation process helped us to discover opportunities for improvement, and address them. We reviewed the project processes and compared them with the Graduated Project criteria defined in the CDF project lifecycle. Based on this review, we made changes in our processes and documentation. It should improve the experience of Jenkins users, and help to make the Jenkins community more welcoming to existing and newcomer contributors.

What changed for the project?

Below you can find a few key changes we have applied during the graduation process:

Public roadmap

We introduced a new public roadmap for the Jenkins project. This roadmap aggregates key initiatives in all community areas: features, infrastructure, documentation, community, etc. It makes the project more transparent to all Jenkins users and adopters, and at the same time helps potential contributors find the hot areas and opportunities for contribution. The roadmap is driven by the Jenkins community and it has a fully public process documented in JEP-14.

More details about the public roadmap are coming next week, stay tuned for a separate blogpost. On July 10th we had an online contributor meetup about the roadmap and you can find more information in its materials (slides, video recording).

User Documentation

• Jenkins Weekly Release line is now documented on our website (here). We have also reworked the downloads page and added guidelines explaining how to verify downloads.

• A new list of Jenkins adopters was introduced on jenkins.io. This list highlights Jenkins users and references their case studies and success stories, including ones submitted through the Jenkins Is The Way portal. Please do not hesitate to add your company there!

Community

• We passed the Core Infrastructure Initiative (CII) certification. This certification helps us to verify compliance with open source best practices and to make adjustments in the project (see the bullets below). It also provides Jenkins users and adopters with a public summary about compliance with each best practice. Details are on the Jenkins core page.

• Jenkins Code of Conduct was updated to the new version of Contributor Covenant. In particular, it sets best practices of behavior in the community, and expands definitions of unacceptable behavior.

• The default Jenkins contributing template was updated to cover more common cases for plugin contributors. This page provides links to the Participate and Contribute guidelines hosted on our website, and helps potential contributors to easily access the documentation.

• The Jenkins Core maintainer guide was updated to include maintenance and issues triage guidelines. It should help us to deliver quality releases and to timely triage and address issues reported by Jenkins users.

What’s next?

It an honor to be the first project to reach the graduated stage in the Continuous Delivery Foundation, but it is also a great responsibility for the project. As a project, we plan to continue participating in the CDF activities and to work with other projects and end users to maintain the Jenkins' leader role in the CI/CD space.

We encourage everyone to join the project and participate in evolving the Jenkins project and driving its roadmap. It does not necessarily mean committing code or documentation patches; user feedback is also very important to the project. If you are interested to contribute or to share your feedback, please contact us in the Jenkins community channels (mailing lists, chats)!

Acknowledgements

CDF graduation work was a major effort in the Jenkins community. Congratulations and thanks to the dozens of contributors who made our graduation possible. I would like to thank Alex Earl, Alyssa Tong, Dan Lorenc, Daniel Beck, Jeff Thompson, Marky Jackson, Mark Waite, Olivier Vernin, Tim Jacomb, Tracy Miranda, Ullrich Hafner, Wadeck Follonier, and all other contributors who helped with reviews and provided their feedback!

Also thanks to the Continuous Delivery Foundation marketing team (Jacqueline Salinas, Jesse Casman and Roxanne Joncas) for their work on promoting the Jenkins project and, specifically, its graduation.

About the Continuous Delivery Foundation

The Continuous Delivery Foundation (CDF) serves as the vendor-neutral home of many of the fastest-growing projects for continuous delivery, including Jenkins, Jenkins X, Tekton, and Spinnaker, as well as fosters collaboration between the industry’s top developers, end users and vendors to further continuous delivery best practices. The CDF is part of the Linux Foundation, a nonprofit organization. For more information about the foundation, please visit its website.

More information

To know more about the Jenkins graduation in the Continuous Delivery Foundation, see the announcement on the CD Foundation website. The press release is available here.

Insurance Data Analytics for Competitive Growth Standards

The insurance industry is data-driven. Data analytics plays a critical role in sales and distribution, fraud detection and prevention, and most importantly in underwriting, risk management, and claims processing. It also provides valuable customer behavior ideas for the insurers to benefit in strategic decision making.Insurance data analytics builds the guidelines for directing better decisions by accurate insights procured from the data analysis. But at the same time, the unpredictable nature of the insurance ecosystem is dependent on the uncertain risk situations and how accurate is the forecasting for the insurance policies and customer outcomes. Predicting such measures needs strong confidence as they are dependent on uncertain factors. Hence, there arises the need for an insurtech based solution that reduces the scope of manual error and only aims for precisely accurate outcomes.So, let us focus on the core functions of insurance analytics to provide insurers with a competitive edge in the industry.Functions of Insurance Data Analytics1. Better Decision-making The strategic function of predictive analytics in insurance is to form a secure, scalable, and well-governed data solution that supports the insurers. The insurance carriers use these data points by connecting them to their complex and rapidly-changing data, integrated with ...


Read More on Datafloq

GitHub Checks API Plugin Project - Coding Phase 2

Another great coding phase for GitHub Checks API Project ends! In this phase, we focused on consuming the checks API in two widely used plugins:

• Warnings NG Plugin

• Code Coverage API Plugin

Besides the external usage, we have also split the general checks API from its GitHub implementation and released both of the plugins:

• Checks API Plugin

• GitHub Checks Plugin

Coding Phase 2 Demo [starts from 25:20]

Warning Checks

The newly released Warnings NG plugin 8.4.0 will use checks API to publish different check runs for different static analysis tools. Without leaving GitHub, users are now able to see the analysis report they interested in.

On GitHub’s conversation tab for each PR, users will see summaries for those checks like the screenshot above. The summaries will include:

• The status that indicates the quality gate

• The name of the analysis tool used

• A short message that indicates statistics of new and total issues

More fine-grained statistics can be found in the Details page.

Another practical feature is the annotation for specific lines of code. Users can now review the code alone with the annotations.

Try It

In Wanings NG plugin 8.4.0, the warning checks is set as a default feature only for GitHub. For other SCM platforms, a NullPublisher will be used which does nothing. Therefore, you can get those checks for your own GitHub project just in a few steps:

1. Update Warnings NG plugin to 8.4.0

2. Install GitHub Checks plugin on your Jenkins instance

3. Follow the GitHub app authentication guide to configure the credentials for the multi-branch project or GitHub organization project you are going to use

4. Use warnings-ng plugin in your Jenkinsfile for the project you configured in the last step, e.g.

node {
    stage ('Checkout') {
        checkout scm
    }

    stage ('Build and Static Analysis') {
        sh 'mvn -V -e clean verify -Dmaven.test.failure.ignore'

        recordIssues tools: [java(), javaDoc()], aggregatingResults: 'true', id: 'java', name: 'Java'
        recordIssues tool: errorProne(), healthy: 1, unhealthy: 20
        recordIssues tools: [checkStyle(pattern: 'target/checkstyle-result.xml'),
            spotBugs(pattern: 'target/spotbugsXml.xml'),
            pmdParser(pattern: 'target/pmd.xml'),
            cpd(pattern: 'target/cpd.xml')],
            qualityGates: [[threshold: 1, type: 'TOTAL', unstable: true]]
    }
}

For more about the pipeline usage of warnings-ng plugin, please see the official documentation.

However, if you don’t want to publish the warnings to GitHub, you can either uninstall the GitHub Checks plugin or disable it by adding skipPublishingChecks: true.

recordIssues enabledForFailure: true, tools: [java(), javaDoc()], skipPublishingChecks: true

Coverage Checks

The coverage checks are achieved by consuming the API in Code Coverage API plugin. First, in the conversation tab of a PR, users will be able to see the summary about the coverage difference compared to previous builds.

The Details page will contain some other things:

• Links to the reference build, including the target branch build from the master branch and the last successful build from this branch

• Coverage healthy score (the default value is 100% if the threshold is not configured)

• Coverages and trends of different types in table format

The pull request for this feature will soon be merged and will be included in the next release of Coverage Checks API plugin. After that, you can use it by adding the below section to your pipeline script:

node {
    stage ('Checkout') {
        checkout scm
    }

    stage ('Line and Branch Coverage') {
        publishCoverage adapters: [jacoco('**/*/jacoco.xml')], sourceFileResolver: sourceFiles('STORE_ALL_BUILD')
    }
}

Like the warning checks, you can also disable the coverage checks by setting the field skipPublishingChecks, e.g.

publishCoverage adapters: [jacoco('**/*/jacoco.xml')], sourceFileResolver: sourceFiles('STORE_ALL_BUILD'), skipPublishingChecks: true

Next Phase

In the next phase, we will turn our attention back to Checks API Plugin and GitHub Checks Plugin and add the following features in future versions:

• Pipeline Support

  • Users can publish checks directly in a pipeline script without requiring a consumer plugin that supports the checks.

• Re-run Request

  • Users can re-run Jenkins build through Checks API.

Lastly, it is exciting to inform that we are currently making the checks feature available on ci.jenkins.io for all plugins hosted in the jenkinsci GitHub organization, please see INFRA-2694 for more details.

Resources

• Checks API Plugin

• GitHub Checks Plugin

• Project Page

• Gitter Channel

Modern Industrial IoT Analytics on Azure – Part 1

This post and the three-part series about Industrial IoT analytics were jointly authored by Databricks and members of the Microsoft Cloud Solution Architecture team. We would like to thank Databricks Solutions Architect Samir Gupta and Microsoft Cloud Solution Architects Lana Koprivica and Hubert Dua for their contributions to this and the two forthcoming posts.

The Industrial Internet of Things  (IIoT) has grown over the last few years as a grassroots technology stack being piloted predominantly in the oil & gas industry to wide scale adoption and production use across manufacturing, chemical, utilities, transportation and energy sectors. Traditional IoT systems like Scada, Historians and even Hadoop do not provide the big data analytics capabilities needed by most organizations to predictively optimize their industrial assets due to the following factors.

Challenge	Required Capability
Data volumes are significantly larger & more frequent	The ability to capture and store sub-second granular readings reliably and cost effectively from IoT devices streaming terabytes of data per day
Data processing needs are more complex	ACID-compliant data processing – time-based windows, aggregations, pivots, backfilling, shifting with the ability to easily reprocess old data
More user personas want access to the data	Data is an open format and easily shareable with operational engineers, data analysts, data engineers, and data scientists without creating silos
Scalable ML is needed for decision making	The ability to quickly and collaboratively train predictive models on granular, historic data to make intelligent asset optimization decisions
Cost reduction demands are higher than ever	Low-cost on-demand managed platform that scales with the data and workloads independently without requiring significant upfront capital

Organizations  are turning to cloud computing platforms like Microsoft Azure to take advantage of the scalable, IIoT-enabling technologies they have to offer that make ingesting, processing, analyzing and serving time-series data sources like Historians and SCADA systems easy.

In part 1, we discuss the end-to-end technology stack and the role Azure Databricks plays in the architecture and design for the industrial application of modern IoT analytics.

In part 2, we will take a deeper dive into deploying modern IIoT analytics, ingest real-time IIoT machine-to-machine data from field devices into Azure Data Lake Storage and perform complex time-series processing on Data Lake directly.

In part 3, we will look at machine learning and analytics with industrial IoT data.

The Use Case – Wind Turbine Optimization

Most IIoT Analytics projects are designed to maximize the short-term utilization of an industrial asset while minimizing its long-term maintenance costs. In this article, we focus on a hypothetical energy provider trying to optimize its wind turbines. The ultimate goal is to identify the set of optimal turbine operating parameters that maximizes each turbine’s power output while minimizing its time to failure.

The final artifacts of this project are:

1. An automated data ingestion and processing pipeline that streams data to all end users
2. A predictive model that estimates the power output of each turbine given current weather and operating conditions
3. A predictive model that estimates the remaining life of each turbine given current weather and operating conditions
4. An optimization model that determines the optimal operating conditions to maximize power output and minimize maintenance costs thereby maximizing total profit
5. A real-time analytics dashboard for executives to visualize the current and future state of their wind farms, as shown below:

The Architecture – Ingest, Store, Prep, Train, Serve, Visualize

The architecture below illustrates a modern, best-of-breed platform used by many organizations that leverages all that Azure has to offer for IIoT analytics.

A key component of this architecture is the Azure Data Lake Store (ADLS), which enables the write-once, access-often analytics pattern in Azure. However, Data Lakes alone do not solve the real-world challenges that come with time-series streaming data. The Delta storage format provides a layer of resiliency and performance on all data sources stored in ADLS. Specifically for time-series data, Delta provides the following advantages over other storage formats on ADLS:

Required Capability	Other formats on ADLS Gen 2	Delta Format on ADLS Gen 2
Unified batch & streaming	Data Lakes are often used in conjunction with a streaming store like CosmosDB, resulting in a complex architecture	ACID-compliant transactions enable data engineers to perform streaming ingest and historically batch loads into the same locations on ADLS
Schema enforcement and evolution	Data Lakes do not enforce schema, requiring all data to be pushed into a relational database for reliability	Schema is enforced by default. As new IoT devices are added to the data stream, schemas can be evolved safely so downstream applications don’t fail
Efficient Upserts	Data Lakes do not support in-line updates and merges, requiring deletion and insertions of entire partitions to perform updates	MERGE commands are effective for situations handling delayed IoT readings, modified dimension tables used for real-time enrichment, or if data needs to be reprocessed.
File Compaction	Streaming time-series data into Data Lakes generates hundreds or even thousands of tiny files.	Auto-compaction in Delta optimizes the file sizes to increase throughput and parallelism.
Multi-dimensional clustering	Data Lakes provide push-down filtering on partitions only	ZORDERing time-series on fields like timestamp or sensor ID allows Databricks to filter and join on those columns up to 100x faster than simple partitioning techniques.

Summary

In this post we reviewed a number of different challenges facing traditional IIoT systems.  We walked through the use case and the goals for modern IIoT analytics, shared a repeatable architecture that organizations are already deploying at scale and explored the benefits of Delta format for each of the required capabilities.

In the next post we will ingest real-time IIoT data from field devices into Azure and perform complex time-series processing on Data Lake directly.

They key technology that ties everything together is Delta Lake. Delta on ADLS provides reliable streaming data pipelines and highly performant data science and analytics queries on massive volumes of time-series data. Lastly, it enables organizations to truly adopt a Lakehouse pattern by bringing best of breed Azure tools to a write-once, access-often data store.

What’s Next?

Learn more about Azure Databricks with this 3-part training series and see how to create modern data architectures by attending this webinar.

--

Try Databricks for free. Get started today.

The post Modern Industrial IoT Analytics on Azure – Part 1 appeared first on Databricks.

Chinese artificial intelligence company files $1.4 billion lawsuit against Apple

Shanghai Zhizhen first sued Apple for patent infringement in 2012 regarding its voice recognition technology. In July, China's Supreme People's court ruled that the patent was valid.

China's ByteDance offers to forgo stake in TikTok to clinch US deal: Report

Under the new proposed deal, ByteDance would exit completely and Microsoft Corp would take over TikTok in the United States.

Reliance Jio to deploy own voice tech for its 5G network

The telco has already developed its own technology which powers its voice over LTE (VoLTE) services nationally and handles more than 10 billion minutes of calls on a daily basis.

Announcing Support for Google BigQuery in Databricks Runtime 7.1

At Databricks, we are building a unified platform for data and AI. Data in enterprises lives in many locations, and Databricks excels at unifying data wherever it may reside. Today, we are happy to announce support for reading and writing data in Google BigQuery within Databricks Runtime 7.1.

Introduction to BigQuery

In Google’s own words, “BigQuery is a serverless, highly scalable and cost-effective data warehouse designed for business agility.” BigQuery is a popular choice for analyzing data stored on the Google Cloud Platform. Under the covers, BigQuery is a columnar data warehouse with separation of compute and storage. It also supports ANSI:2011 SQL, which makes it a useful choice for big data analytics.

Enhancements for Databricks users

The Spark data source included in Databricks Runtime 7.1 is a fork of Google’s open-source spark-bigquery-connector that makes it easy to work with BigQuery from Databricks:

• Reduced data transfer and faster queries: Databricks automatically pushes down certain query predicates, e.g., filtering on nested columns to BigQuery to speed up query processing and reduce data transfer. These optimizations are automatically applied to your queries.
• Direct query: Transforming and filtering the data residing in a BigQuery table using existing Spark APIs can first mean transferring large amounts of data from BigQuery to Databricks. To reduce data transfer costs, we have added the capability to first run a SQL query on BigQuery with the query() API and only transfer the resulting data set.

Examples

The following examples show how easy it is for BigQuery users to get started with Databricks.

Read the results of a BigQuery SQL query into a DataFrame

val table = "bigquery-public-data.samples.shakespeare"
val tempLocation = "databricks_testing"

// read the entire table into a DataFrame
val df1 = spark.read.format("bigquery").option("table", table).load()

// read the result of a BigQuery SQL query into a DataFrame
val df2 =
        spark.read.format("bigquery")
        .option("materializationDataset", tempLocation)
        .option("query", s"SELECT count(1) FROM `${table}`")
        .load()
        .collect()

Write a DataFrame to a BigQuery table

df.write
    .format("bigquery")
    .mode("append")
    .option("temporaryGcsBucket", tempLocation)
    .option("table", "mycompany.employees")
    .save()

Use cases

Support for BigQuery will enable new use cases, including these examples that our customers are already building:

• Advanced analytics and machine learning on data stored in Google Cloud: Take advantage of the power of Databricks’ collaborative data science environment to supercharge the productivity of your data teams. You can also standardize the ML lifecycle from experimentation to production and enable ML and AI on data in Google Cloud.
• Multi-cloud data integration: If part of your data resides in Google Cloud, you can use Databricks to bring together data silos and unlock the full value of your data.

See the documentation for detailed information on how to get started.

--

Try Databricks for free. Get started today.

The post Announcing Support for Google BigQuery in Databricks Runtime 7.1 appeared first on Databricks.