Thursday, 5 March 2020

A Look into the Mid Market Sales Team

At Databricks, we are passionate about helping data teams solve the world’s toughest problems. Databricks helps organizations innovate faster and tackle challenges like treating chronic disease through faster drug discovery, improving energy efficiency, and protecting financial markets. With the help of our sales team, we are able to connect thousands of global customers to Databricks’ Unified Data Analytics Platform, and help them with their mission-critical workloads. Learn more from Jules Gsell, Sr. Director of Mid Market Sales, on what the Mid Market Sales team does and how they impact Databricks.

The Databricks Mid Market Sales team

Jules (front row, on the left) with the Mid Market Sales team

Tell us a little bit about yourself and the team you lead.

I lead the Mid Market Sales team for North America at Databricks. The Mid Market Sales team focuses on helping digital native start-ups, many of whom are not yet household names. These customers come from all industries and have incredible missions powered by Databricks.

I’ve been with Databricks just over 2 years and have been selling or leading sales teams in tech for the last 11 years. When I am not helping our customers solve the world’s toughest data problems, I am hanging out with my husband, son, and our golden retriever. My personal interests include running, interior design, and travel.

What were you looking for in your next opportunity, and why did you choose Databricks?

I always knew I wanted to be closer to the product side of the organization and our customer’s customer. Helping companies meet their goals when it comes to customer experience and competitive differentiation was a really exciting opportunity. In addition to this, I knew I wanted to go back to an earlier stage company and be able to make an impact on the sales strategy as well as the culture.

I picked Databricks largely because of the leadership team and the direct impact we have on our customers’ business. I also knew I would be challenged and grow through the process; I tend to get bored easily, so I am always looking to push my own limits.

What does your day-to-day look like?

I strive to spend as much of my time as possible with my team on customer calls. I would spend 100% of my time on this if I could, as I love hearing from our customers and guiding my team on how to best support our customers. Outside of this, I spend a large amount of my time in one-on-one’s with my team, meeting potential candidates and aligning on our strategy with my peers, both internal like Customer Success and external, such as ISV and cloud partners.

At Databricks, we believe that “teamwork makes the dream work”. Which teams do you and the Mid Market Sales team collaborate with the most and how do you work together?

We work very closely with our Solution Architecture team in the pre-sales process as they are the experts and key to us demonstrating our value to prospective customers. Being closely aligned with them on our strategy is so instrumental. We also work very closely with post sales and our entire Customer Success organization who ensure our customers know how committed we are to their success and delivering on their mission. We could not do our jobs without either of these teams.

One of our core values at Databricks is to be an owner. What are some of the most memorable moments when the Mid Market Sales Team at Databricks owned it?

The Databricks Mid Market team at a team outing hike in San Francisco

The Mid Market team at a team outing hike in San Francisco

In sales, you own it everyday! I challenge my team to focus on what is in our direct control and hold ourselves accountable to the outcomes we can drive. When we focus on that, it simplifies a lot.

I am particularly proud of our team for driving engagement with our customers. It’s one thing when customers start using our platform, it’s a much bigger project to drive digital transformation across the organization. Most of our customers are born in the cloud but many of them are still struggling to unify their analytics across their data teams, which live across siloed teams and infrastructure. Once we have the trusted advisor relationship, we can really jump in and help them identify areas that streamline their workflow and help the broader organization be more successful.

Every day presents a growth opportunity. Our customers are some of the most innovative organizations in the world and they are always bringing new ideas and opportunities to us. We have customers who are disrupting the disruptors, from companies improving the life of a truck driver, accelerating drug discovery through genomics research, and disrupting the pay cycle to enable the hourly worker to save money and reduce debt. It is my honor to work with these companies every day and innovate alongside them.

What is some advice you’ve shared with your team to help them grow in their careers?

In my career, I have learned that there are three things that are very important to your success no matter what your role is in the organization. First, treat everyone with kindness, honesty and respect. This should be a given but I have learned that it cannot be underestimated. You will earn the trust of your customers, peers, leaders and employees and trust is critical to collaboration. Second, learn to be an advocate for yourself and seek out mentorship. I recommend carving out 30 minutes per week with cross-functional peers or leaders to learn about their organization and how you can help and share some of the relevant projects and milestones you are already working on. Third, step out of your comfort zone. Looking back, I have grown the most when I did something I was not entirely comfortable doing yet. It can be nerve wracking but it is important to be challenged and seek out opportunities to do so.

Want to work on the Mid Market Sales team with Jules? Check out our Careers Page.

--

Try Databricks for free. Get started today.

The post A Look into the Mid Market Sales Team appeared first on Databricks.

I Joined Databricks to Make Data Science a Little Less Scary

Big data and AI has always struck me as useful, but slightly scary.

For example, it’s useful when Waze uses big data to help me outsmart a traffic jam. On the other hand, big data’s ad-targeting is so powerful that millions of people are scared that their smartphones are eavesdropping on them.

Accessible Design is good design and helps make the complex technology behind things like big data and artificial intelligence be more useful and accessible to a wider audience.

Why Big Data and AI Is Intimidating to So Many People

The reason so many of us think big data is scary is because it’s opaque and powerful and we don’t have a way to creatively participate in it. In order to make big data and artificial intelligence truly useful and less opaque, data science tools need to be easier to use and accessible to more of us. Great design is one way to accomplish this.

Databricks is a technical company that is committed to democratizing the power of big data. While it has developed extremely powerful tools, the company is also a big believer in harnessing the power of design to make big data analysis and AI easier for people everywhere. Today, I am thrilled to announce that I’ve decided to join Databricks as VP of Product Design to help achieve this goal.

While technical requirements such as Python and statistical modeling aren’t going away anytime soon, good design can radically reduce the complexity of data science tools and support a more diverse range of technical ability. In fact, job market data suggests that easier-to-use data science tools are going to be the “next big thing.”

How Good Design Can Make Data Science More Accessible

In August 2018, LinkedIn reported that there’s a shortage of 151,717 people with data science skills in the U.S., based on data from its platform. Better-designed and easier-to-use data science tools will be key to filling this gap.

According to a recent Gartner study, by 2022 it’s predicted that 40% of machine learning model development and scoring will occur outside of tools where machine learning is the primary job-to-be-done. That means that other forms of less-technical business software will become smarter and enable more of us to participate in the predictive powers of machine learning.

For another example of how quickly the barrier-to-entry for machine learning and big data tools is dropping, look at biotech. In the very recent past, the time and cost to do machine learning on genetic data from hundreds of thousands of people was prohibitively expensive. It took months for biological researchers to run a single analysis, and they would often have to restrict their hypotheses to a basic set of questions that they could answer with small datasets using only a couple hundred people. Querying across hundreds of thousands of humans would have required a massive team of data scientists and data engineers, if it was even possible.

Today what was impossible has become the new normal. Performing ML on a massive dataset, like genetic data from a human population, has become dramatically easier and more affordable. Just the other day, a pharma company called Regeneron announced that it would be partnering with the US government to deliver a vaccine for the Coronavirus in a couple of months time. Previously, this would have taken several years.

Regeneron isn’t the only company using Databricks to design solutions to the world’s hardest problems.

A package of medical supplies delivered via a Zipline drone
A package of medical supplies delivered via a Zipline drone (photo: Zipline International)

Zipline is an on-demand drone delivery company that delivers medicine and blood to clinics in rural Africa. Zipline flies so many drone missions each year, their fleet can quickly generate large data sets. This data can be mined inside Databricks for important insights and business intelligence that help Zipline optimize its operations. This is another example of how big data can literally help save lives.

As a designer, I’ve always loved working with great entrepreneurs to solve big problems and I feel like the decision to join Databricks is the start of a meaningful and impactful journey as a problem-solver. Using the power of great design, Databricks is beginning a journey to make AI and big data easier to manage so that more people can work on solving the world’s hardest problems. And with the mountain of problems facing humanity, it would seem we have little time to waste.

If you’re serious about the potential of design to play a huge role in making AI and big data tools less scary and more accessible, I hope you will join me.

Ryan is the VP of Product Design at Databricks. Find Ryan’s original article on Medium: I joined Databricks to Make Data Science a Little Less Scary.

--

Try Databricks for free. Get started today.

The post I Joined Databricks to Make Data Science a Little Less Scary appeared first on Databricks.

How to Overcome Internal Resistance to Process Automation and Outsourcing

Business Process Outsourcing (BPO) and Robotic Process Automation (RPA) are two emerging trends that are allowing organizations to become leaner, more flexible and efficient. Both innovations achieve similar results for an organization by freeing up key players from non-productive tasks and allowing them to focus on core business activities.Unfortunately, both innovations also encounter the same problems when first being introduced; the resistance from employees who distrust change and fear such measures will make their own jobs obsolete. Here are some creative ways companies can overcome resistance to BPO and RPA. Business Process OutsourcingBusiness Process Outsourcing (BPO) is where an organization contracts out certain non-primary business activities to a third party service provider.BPO services can include payroll, human resources, accounting, customer service, and document management services, all of which can be performed faster and more cost-effectively by a BPO provider.Outsourcing also gives an organization access to skilled manpower on an as-needed basis, allowing it to leverage needed expertise without having to take on extra employees.Robotic Process Automation (RPA)Robotic Process Automation (RPA) is the process of automating business operations using robots to reduce the need for human intervention.Involving either actual robots or software that automates business activities, RPA offers a range of ...


Read More on Datafloq

The Role Of Artificial Intelligence In Healthcare

Artificial Intelligence and Machine Learning are the two tech components that are continuously transforming various industries particularly the healthcare sector. Considering the uncountable benefits AI can bring to the sector, many leading brands and companies are investing in bringing out innovations using AI. According to the CB insights report of 2016, by the year 2020, companies are going to spend around $54 million on AI-projects.


The AI Robotics is undoubtedly providing the greatest support to bringing efficiency in healthcare at the lowest cost. Being a source to eliminate or diagnose many severe diseases, it has rooted itself into the industry strongly.Some Of The Great Examples of AI in UseLets learn about the many successful application of AI and how its benefiting the doctors and patients simultaneously. The AI-Assisted Treatment Center, ChinaChina has established its first-ever AI-assistant treatment center. The center is a part of the bigger plan. China wants to contribute around 1 trillion dollars to the healthcare industry and with this center;it's going to achieve its ultimate goal. This treatment center has robots who assist the surgeons and specialist in carrying out a different task and running diagnoses. ...


Read More on Datafloq

Wednesday, 4 March 2020

Leveraging Technological Advances to Facilitate Online Trading

The online trading marketplace continues to grow rapidly amid rapid advancements in technology. Traditionally, traders would either learn to trade gradually or hire an expert trader to trade on their behalf. This limited the inflow of new online traders into the market.However, with the latest advances in technology, traders can now leverage various trading tools and systems to make money online without necessarily seeking the advice of an expert, or becoming experts themselves. Online trading brokers have also recognized the paradigm shift and moved quickly to provide tools that will bring in more traders. As such, some offer their traders an opportunity to automatically copy the trades of expert traders, while others provide interactive charting tools to help analyze the markets.Adapting trading platforms to mobile devicesNearly every major city and town in the world today have access to the internet. The rise in global internet penetration has been driven by the emergence of the smartphone. Over the last 10 years, communication has continued to shift from e-mail to mobile messaging apps with billions now using Facebook’s Messanger App, Instagram, WhatsApp, Chinese WeChat, TikTok, and Telegram among others.Investment service providers have followed the changes in consumer behavior by developing online investment ...


Read More on Datafloq

ISRO calls off GISAT-1 launch due to technical reasons

The Indian Space Research Organisation said the “revised launch date will be informed in due course” without disclosing specific details on the technical reasons or glitches. The satellite was one of the key launches planned by the country’s space agency for this year.

Data Quality Monitoring on Streaming Data Using Spark Streaming and Delta Lake

Elaborating the true cost of compliance

Compliance was something non-existent before the 20th century and was termed as obeying. It has now become really vital in everyday life as the rise in global terrorism wave has transformed the word compliance from rarity to a completely new industry in the digital world. Compliance is one of the main pillars in almost every domain.Compliance was first used after it was made obligatory by regulators for entities to follow and optimize their operations according to the regulator's guidelines in the mainstream industry. These guidelines were issued after the foundation of the Financial Crimes Enforcement Network in 1990. After 9/11 the anti-money laundering & terror financing gave the world a whole new united agenda to comply with the latest regulations made to combat terrorism. This led toward the creation of regulatory compliance to monitor and layout specific guidelines to prevent money laundering around the globe. And focus on creating a risk-free environment. Before you bash me up, risk is something very often associated with compliance but they are not really correlated yet can’t go without each other.The reason is Compliance is a part of risk mitigation and deployed to get total risk reduced at a drastic level. How compliance and ...


Read More on Datafloq

Banking ban on cryptocurrencies considered disproportionate by SC

Industry bodies and founders of cryptocurrency exchanges have said that the verdict would help in shaping discussions on regulating cryptocurrency trade in the country and opening up the Indian market to global crypto trade.

Facebook weighs Libra revamp to address regulatory concerns

The overhaul is intended to ensure that the new payments network would be compatible, rather than in competition, with those projects.

Getting into Big Data Career: An Overview

Big data is all about what organizations do with the massive amount of data collected on a day-to-day basis. The data gets collected at a faster rate in varied forms and large volumes. It is used by organizations and businesses to discover patterns and trends to understand human behavior. It helps to know how humans interact with technology, what products or information he consumes, and many more. These data are further used to make decisions for human benefit and profit. Scientists, medical practitioners, governments, advertisers, marketers, and businesses are using big data. The ability to slice and dice big data facilitates predictive analytics and user behavior analytics. The companies need the best talent as a big data job involves analyzing data, extracting systematic information, and dealing with large or complex datasets. The analytics industry has grown to $3.03 billion in size in 2019 and is expected to double by 2025 as per Analyticsindiamag.com reports. Big data specialists will be trending by the year 2022 according to the World Economic Forum. The Dice 2020 Tech Job Report labeled data engineer as the fastest-growing job in 2019 with a 50% year-over-year growth in the number of positions. Big data has made the ...


Read More on Datafloq

SC strikes down RBI's crypto ban; calls it unconstitutional

Supreme Court had held the RBI's ban on virtual currencies as unconstitutional in the case of IAMAI vs RBI.

What is the most prolific cyber threat from IoT devices?

IoT varies from common traditional internet since it doesn’t rely on human intervention to function. It has developed by leaps and bounds in a defined time frame. Ranging from the development of smartwatches to washing machines with internet connectivity, more gadgets and appliances now connected in some form to the internet. These devices are designed to do everything from boosting productivity of a company, reducing energy spending within a house, and lowering overhead. More businesses are now expected to explore latest ways to deploy IoT devices. However, there are skilled hackers and cyber-criminals taking IoT devices as a backdoor into a company network. The strength of network security is defined by the weakest link that lies inside an IoT connected device. Any enterprise should understand the impact of IoT on cybersecurity as nobody from the company and its clientele will be glad to know when confidential details and financial documents breach. The Concept of IoTFirst, understand what IoT is. IoT is any inter-networking of a device having internet connectivity for exchanging data. IoT enabled devices can be smartwatches, hardware sensors, computer programs, etc. Modern terms define IoT as smart devices. So, any gadget with smart pre-fix or tag is an ...


Read More on Datafloq

Startup Boom and the Rapid Rise in Co-Working Spaces

Over the last three years, shared workspaces have grown from just over 2% in 2017 to nearly 8% of the global rental office space. This growth has been attributed to several factors, but startups top that list.Ideally, shared office spaces provide workers with a fully-services workspace that comes loaded with free internet, mailing services, on-demand boardroom access, a desk and a chair, and many more. To some people, it would appear that these add-ons are the driving force behind the increasing demand for co-working spaces.However, according to selected surveys, there are far better benefits than free internet and mailing services. In fact, these add-ons are technically not free because they are covered in the rental fees. Depending on the package you choose, the list of add-ons can be shorter or longer as seen in offices.net, a shared office agency platform. This shows that at the end of the day, they are paid for.Why people like shared workspacesBut there is more to working in a shared office space. According to an infographic published by Visual Capitalist, the ability to interact with others in a community of like-minded individuals is one of the primary forces behind the rapid growth of shared workspaces. ...


Read More on Datafloq

Findsecbugs for Developers

Spotbugs is a utility used in Jenkins and many other Java projects to detect common Java coding mistakes and bugs. It is integrated into the build process to improve the code before it gets merged and released. Findsecbugs is a plugin for Spotbugs that adds 135 vulnerability types focused on the OWASP TOP 10 and the Common Weakness Enumeration (CWE). I’m working on integrating findsecbugs into our Jenkins ecosystem.

Background

Spotbugs traces its history through Findbugs, which started in 2006. As Findbugs it was widely adopted by many projects. About 2016, the Findbugs project ground to a halt. Like the mythical phoenix, the Spotbugs project rose from the ashes to keep the capabilities alive. Most things are completely compatible between the two systems.

Jenkins has used Findbugs and now Spotbugs for years. This is integrated as a build step into parent Maven poms, including the plugin parent pom and the parent pom for libraries and core components. There are various properties that can be set to control the detection threshold, the effort, and findings or categories to exclude. Take a look at the effective pom for a project to see the settings.

Conundrums

There is a fundamental conundrum with introducing an analysis tool into a project. The best time to have done it is always in the past, particularly when the project first started. There are always difficulties in introducing it into an existing project. Putting it off for later just delays the useful results and makes later implementation more difficult. The best time to do it is now, as early as possible.

All analysis tools are imperfect. They report some issues that don’t actually exist. They miss some important issues. This is worse in legacy code, making the adoption more difficult. Findings have to be examined and evaluated. Some are code weaknesses but don’t indicate necessary fixes. For example, MD5 has been known for years as a weak algorithm, unsuitable for security uses. It can be used for non-security purposes, such as fingerprinting, but even there other algorithms (SHA-2) are preferred. We should replace all usages of MD5, but in some cases that’s difficult and it’s not exactly a problem.

Ultimately, the gain from these analysis tools isn’t so much from finding issues in existing code. The value comes more from catching new regressions that might be introduced or improving new code. This is one reason why it is valuable to add useful new analysis such as findsecbugs now, so that we can begin reaping the benefits.

With a security tool like findsecbugs, there is another paradox. Adding the tool makes it easier to find potential security issues. Attackers could take advantage of this information. However, security by obscurity is not a good design. Anyone can run findsecbugs now without the project integrating it. Integrating it makes it easier for legitimate developers to resolve issues and prevent future ones.

Implementation

I’ve been working on integrating findsecbugs into the Jenkins project for several months. It is working in several repos. There are several others where I have presented draft PRs to demonstrate what it will look like once it is enabled. As soon as we can disseminate the information enough, I propose to enable it in the parent poms for widespread use.

Existing

I started by enabling findsecbugs in two major components where I have a high degree of familiarity, Remoting, and Jenkins. Most of the work here involves examining each finding and figuring out what to do with it. In most cases this results in using one of the suppression mechanisms to ignore the finding. In some cases, the code can be removed or improved.

Findsecbugs reported a significant number of false positives in Remoting for a couple of notable reasons. (See the PR.) Remoting uses Spotbugs aggressively with a Low threshold setting. This produces more results. Findsecbugs targets Java web applications. As the communication layer between agents and master, Remoting uses some mechanisms that would be a problem on the server side but are acceptable on the agent.

Even without all its plugins, Jenkins is a considerable collection of code. Findsecbugs reported a smaller number of false positives for Jenkins (See the PR.) It runs Spotbugs at a High threshold, so it only reports issues it deems more concerning. A number of these indicate code debt, deprecated code to remove, or areas that could be improved. I created Jira tickets for many of these.

Demonstrated

I have created draft PRs to demonstrate how findsecbugs will look in several plugins. The goal is not to use these PRs directly but instead integrate findsecbugs at the parent pom level. These PRs serve as reference documentation.

Credentials

This one is particularly interesting because here findsecbugs correctly detects the remains of a valid security vulnerability (CVE-2019-10320). Currently, this code is safely used only for migration of old data. If we had run findsecbugs on this plugin a year ago, it would have detected this valid vulnerability.

SSH Build Agents

This one is interesting because it flags MD5 as a concern. Since it is used for fingerprinting, it isn’t a valid vulnerability, but since the hash isn’t stored it is easy to improve the code here.

EC2

In this case, findsecbugs found some valid concerns, but the code isn’t used so it can be removed. Also, MD5 is harder to remove here but should be considered technical debt and removed when possible.

Platform Labeler

Findsecbugs didn’t find any concerns here. This means adapting to it requires no work. In this demonstration, I added a fake finding to prove that it was working.

File Leak Detector

There is one simple finding noted here. Because it is part of the configuration performed by an administrator we can ignore it.

Credentials Binding

Nothing was found here so integration requires no effort.

Proposed

My proposal is to integrate findsecbugs configuration into the parent poms as soon as we can. The delay is currently mostly around sharing the information to prepare developers by blog post, email list discussion, and presentation.

Even before I started working on this, StefanSpieker proposed a PR to integrate into the parent Jenkins pom. This will apply to Jenkins libraries and core components. Once this is integrated, I will pull out the changes I made to the Jenkins and Remoting project poms.

I also plan on integrating findsecbugs into the plugin and Stapler parent poms. Once it is added to the plugin parent pom all plugins will automatically perform these checks when they upgrade their parent pom version. If there are any findings, developers will need to take care of them as described in the next section.

What do you need to do?

Once developers upgrade to a parent pom version that integrates findsecbugs, they may have to deal with evaluating, fixing, or suppressing findings. The parent pom versions do not yet exist but are in process or proposed.

Extraneous build message

In some cases, an extraneous message may show up in the build logs. It starts with a line like this The following classes needed for analysis were missing: followed by lines listing some methods by name. Ignore this message. It results from SpotBugs printing some internal, debug information that isn’t helpful here.

Examine findings

If findsecbugs reports any findings, then a developer needs to examine and determine what to do about each one.

Excluding issues

You can exclude an issue, so that it is never reported in a project. This is done by configuring an exclusion file. If you encounter the findings CRLF_INJECTION_LOGS or INFORMATION_EXPOSURE_THROUGH_AN_ERROR_MESSAGE feel free to add these to an exclusion file. These are not considered a concern in Jenkins. See the Jenkins project exclusion file for an example. You should be cautious about including other issue types here.

Temporarily disable findsecbugs

You may disable findsecbugs by adding <Bug category="SECURITY"/> to the exclusion file. I strongly encourage you to only disable findsecbugs temporarily when genuinely needed.

Suppress a finding

After determining that a finding is not important, you can suppress it by annotating a method or a class with @SuppressFBWarnings(value = “…​”, justification=”…​”). I encourage you to suppress narrowly. Never suppress at the class level when you can add it to a method. For a long method, extract the problematic part into a small method and add the suppression there. I also encourage you to always add a meaningful justification.

Improve code

Whenever possible improve the code such that the problematic code no longer exists. This can include removing deprecated or unused code, using improved algorithms, or improving structure or implementation. This is where the significant gains come from with SpotBugs and findsecbugs. Also, as you make changes or add new features make sure to implement them so as not to introduce new issues.

Report security vulnerabilities

If you encounter a finding related to a valid security vulnerability, please report it via the Jenkins security reporting process. This is the responsible behavior that benefits the community. Try not to discuss or call attention to the issue before it can be disclosed in a Jenkins security advisory.

Create tasks

If you discover an improvement area that is too large to fit into your current work or release plan, I encourage you to record a task to get it done. You can do this in Jira, like I did for several issues in Jenkins core, or in whatever task management system you use.

Conclusion

SpotBugs has long been used in Jenkins to catch bugs and improve code quality. Findsecbugs adds valuable security-related bug definitions. As we integrate it into the existing Jenkins code base it will require analysis and suppression for legacy code. This identifies areas we can improve and enhances quality as we move forward. Please responsibly report any security vulnerabilites you discover.

The Integration of Billboard Advertising and Digital Marketing

Advertising is the concept of creating awareness about a brand, product, or event. Its application has evolved over the years from simple paper posters on the streets to wooden boards, billboards, radio, TV and now online digital advertising. While some of the methods used to advertise in the past continue to fade, some remain relatively important to businesses looking to launch a successful marketing campaign. Billboard advertising has evolved over the years constantly adapting to new technological advances. We have moved from the traditional led tubes to more energy-efficient neon signs which businesses have found to be relatively effective compared to their counterparts. However, even with this kind of evolution, businesses still need to integrate their billboard marketing activities into the new age of digital marketing. This has proved to be a little challenging for many startups that cannot afford the services offered by capable billboard marketing companies.Billboard marketing agencies utilizing intuitive analytics softwareTop technology players have increased their usage of intuitive business analytics tools to provide live analysis of the performance of ads placed on billboards at strategic locations. This technology provides clients with an opportunity to use the information provided by the analytics platform to improve their digital ...


Read More on Datafloq

Tuesday, 3 March 2020

Cybersecurity in the Year of the RAT

According to the Chinese zodiac, 2020 is the year of the rat. The first of the astrological animals, the cycle of 12 has begun again, and similarly, this year is associated with the opportunity for new beginnings. However, from a technological perspective, it seems that 2020 is also the year of the RAT (Remote Access Trojan), an opportunity for cybercriminals to visit chaos upon our lives all over again. This year is likely to see a deeper integration of technology into more aspects of our lives. Between objects in the Internet of Things becoming more affordable to 5G networks beginning to establish a presence in our communities, there are certainly interesting areas to explore. But this also means that there is greater scope for unscrupulous actors to introduce malware, ransomware, and other attacks to our systems. So what can we expect from cybersecurity in 2020? What are the potential areas of vulnerability, and how can we best address them? In a year that is intended to be one of new beginnings, how can we avoid falling at the first hurdle? The ThreatsLet’s cut to the chase — what are the primary cybersecurity threats we’re facing this year? Well, the ...


Read More on Datafloq

How Drupal and Artificial Intelligence Together Drive Better Personalization

The emergence of artificial intelligence and its continued evolution ever since has had significant ramifications for the world, especially in the context of technology and how we use it. Among the virtually endless number of things that artificial intelligence is used for, one is healthcare. Yes, we live in a world where artificial intelligence has a proven track record of enabling significantly better diagnosis and treatment of the diagnosed ailments. See, the point is that this technology’s potential is so vast that its applications in the real world continue to grow at an equally rapid pace. It is why artificial intelligence has made inroads into the interwebs
and enterprise CMS as well. But before we can delve any deeper, it is
essential to identify that Drupal is the leading choice of CMS across
enterprises. Now that we have cleared that up let’s see what the
combination of Drupal and artificial intelligence can do. One of the
most compelling benefits obtained from the coming together of Drupal and
this nifty technology is the ability to glean insights from all the
data gathered and involved in the process. Read on to see what else this
duo can help with.1. Personalization: Content tailored ...


Read More on Datafloq

The 7 Ecommerce Trends You Can’t Afford to Miss Out On in 2020

A study published by eMarketer in 2016 predicted a double-figure increase in the eCommerce sector, topping out at an estimated $4.058 trillion by 2020.


Not too many industries have such a rosy outlook. In this article, we’ll take a look at some of the initiatives that are likely to take the eCommerce industry by storm, allowing you to remain a step ahead of both the technology curve and your competition.#1. Quicker Shipping and Improved Logistics


Image source: Andrea Piacquadio from PexelsWhile convenience and relative ease of use are the domain of eCommerce, brick and mortar stores have an advantage that shopping sites don’t: immediate availability of purchased items. Typically, buying from an eCommerce platform means having to wait a few days for your item to be delivered.However, that’s about to change. While generally tight-lipped about its performance metrics, Amazon ships 2.5 billion packages per year throughout the world using its Amazon Prime and the free one- or two-day delivery commitment. These numbers were just released in 2019, and there is no reason to believe that things would ...


Read More on Datafloq

What Essential Skills and Areas of Knowledge Should a Data Science Specialist Have?




Data science is a vast field that combines several related disciplines. These are programming, mathematics & statistics, business analytics, and machine learning. Specialists in the field of data science are data analysts, who work with large amounts of data by extracting useful information for decision making in business. The result provides answers to many questions: for example, how many units of goods should be purchased in the next quarter, and which component of the medicine will improve the patient’s well-being. To solve such problems, specialists develop algorithms that are able to generate a result without human intervention.What you need to understandThe requirements for training and the level of professional skills depend on which company the specialist will work in. For instance, in large corporations, data analytics is important to understand mathematics and statistics. Whereas, marketplaces and media companies need experts in developing recommender systems.Following are the skills and areas of knowledge that a successful specialist will need.ProgrammingThe most popular and common language in Data Science today is Python. Earlier, the most popular language was R, which is still used for data analysis, scientific statistical analysis, and also in ...


Read More on Datafloq

Why You Should Prefer Cryptocurrencies

Ever since the Bitcoin made its popularity all over the world, it has generated interest in cryptocurrency among many people. The major reason behind it is the elimination of the role of the middleman. Also, it is not controlled by any central authority like banks or even government. The rise of Bitcoin paved the way to many other alternate virtual currencies, which are called as altcoins. Although, they may not be at that superior level to Bitcoin, still, it does have to compete with these multiple cryptocurrencies. So today we will be delving into know about why you should prefer cryptocurrencies.If you are abreast with cryptocurrency latest news today, then you will know the global economy updates and how well it is moving further to digital eco-system. Be it money transfer or investing, everyone prefers paperless technology. Cryptocurrency is one such latest trend which is becoming the promising and existing add on to the digital payments. If you are a budding entrepreneur who wants to deal in cryptocurrencies in future, then better accumulate knowledge about this newest sensation in the contemporary financial world. What Does Cryptocurrency Mean?Unlike normal currencies including USD, cryptocurrencies deals in exchange of information via digitally. In ...


Read More on Datafloq

Google's self-driving unit Waymo raises $2.25B in its first external investment

Among the investors include auto parts supplier Magna International, U.S. dealership chain AutoNation, the Canada Pension Plan Investment Board, and investment firms Silver Lake, Andreessen Horowitz and Mubadala Investment.

Indian EV industry looks for a way around Covid-19

Analysts say manufacturers like Tata Motors, Bajaj Auto and TVS are evaluating their options to mitigate the impact of the outbreak

Monday, 2 March 2020

Scrum, Objectives and Key Results

From books - we believe that 'Agile' is referred to as a set of practices or methods which are based on the values and principles. Agile helps to improve communication among self-organized, cross-functional teams working in a collaborative environment. 'Scrum' is a framework used to implement Agile development. A Scrum Team is a collection of individuals working together to deliver the required product increments. The Scrum team includes the Product Owner, Scrum Master, and Development team.A few months back, I received a chance to attend a Scrum Master training program and I should say, it had nearly changed my idea on responsibilities for this post. Till then, I was in a dilemma that SM (Scrum Master) was a team role, responsible for ensuring the team lives in agile values and principles and follows different processes anywhere in the project (at the beginning, towards the end, in documents or like as we all do, while pitching the project at sales point). Some of my previous scrum projects didn't have stand-ups, retrospectives, etc. All team members were in one particular view - complete the project in the provided time. Later, I came across some masters in this field, who shared their thoughts ...


Read More on Datafloq

Scrum, Objectives and Key Results

From books - we believe that 'Agile' is referred to as a set of practices or methods which are based on the values and principles. Agile helps to improve communication among self-organized, cross-functional teams working in a collaborative environment. 'Scrum' is a framework used to implement Agile development. A Scrum Team is a collection of individuals working together to deliver the required product increments. The Scrum team includes the Product Owner, Scrum Master, and Development team.A few months back, I received a chance to attend a Scrum Master training program and I should say, it had nearly changed my idea on responsibilities for this post. Till then, I was in a dilemma that SM (Scrum Master) was a team role, responsible for ensuring the team lives in agile values and principles and follows different processes anywhere in the project (at the beginning, towards the end, in documents or like as we all do, while pitching the project at sales point). Some of my previous scrum projects didn't have stand-ups, retrospectives, etc. All team members were in one particular view - complete the project in the provided time. Later, I came across some masters in this field, who shared their thoughts ...


Read More on Datafloq

Scrum, Objectives and Key Results

From books - we believe that 'Agile' is referred to as a set of practices or methods which are based on the values and principles. Agile helps to improve communication among self-organized, cross-functional teams working in a collaborative environment. 'Scrum' is a framework used to implement Agile development. A Scrum Team is a collection of individuals working together to deliver the required product increments. The Scrum team includes the Product Owner, Scrum Master, and Development team.A few months back, I received a chance to attend a Scrum Master training program and I should say, it had nearly changed my idea on responsibilities for this post. Till then, I was in a dilemma that SM (Scrum Master) was a team role, responsible for ensuring the team lives in agile values and principles and follows different processes anywhere in the project (at the beginning, towards the end, in documents or like as we all do, while pitching the project at sales point). Some of my previous scrum projects didn't have stand-ups, retrospectives, etc. All team members were in one particular view - complete the project in the provided time. Later, I came across some masters in this field, who shared their thoughts ...


Read More on Datafloq

Scrum, Objectives and Key Results

From books - we believe that 'Agile' is referred to as a set of practices or methods which are based on the values and principles. Agile helps to improve communication among self-organized, cross-functional teams working in a collaborative environment. 'Scrum' is a framework used to implement Agile development. A Scrum Team is a collection of individuals working together to deliver the required product increments. The Scrum team includes the Product Owner, Scrum Master, and Development team.

A few months back, I received a chance to attend a Scrum Master training program and I should say, it had nearly changed my idea on responsibilities for this post. Till then, I was in a dilemma that SM (Scrum Master) was a team role, responsible for ensuring the team lives in agile values and principles and follows different processes anywhere in the project (at the beginning, towards the end, in documents or like as we all do, while pitching the project at sales point). Some of my previous scrum projects didn't have stand-ups, retrospectives, etc. All team members were in one particular view - complete the project in the provided time. Later, I came across some masters in this field, who shared their thoughts ...


Read More on Datafloq

Sunday, 1 March 2020

Pipeline-Authoring SIG Update

What is the Pipeline-Authoring Special Interest Group

This special interest group aims to improve and curate the experience of authoring Jenkins Pipelines. This includes the syntax of `Jenkinsfile`s and shared libraries, code sharing and reuse, testing of Pipelines and shared libraries, IDE integration and other development tools, documentation, best practices, and examples.

What Are The Focus Areas of the Pipeline-Authoring Special Interest Group

  • Syntax - How `Jenkinsfile`s and shared libraries are written.

  • Code sharing and reuse - Shared libraries and future improvements.

  • Testing - Unit and functional testing of `Jenkinsfile`s and shared libraries.

  • IDE integration, editors, and other development tools - IDE plugins, visual editors, etc.

  • Documentation - Reference documentation, tutorials, and more.

  • Best practices - Defining, maintaining, and evangelizing best practices in Jenkins Pipeline.

  • Examples - Real-world `Jenkinsfile`s and shared libraries demonstrating how to utilize various features of Pipeline, as well as basic or starter `Jenkinsfile`s for common patterns that can be used as jumping-off points by new users.

What Have We Been Up To

With the start of a new year, members got together to discuss the roadmap for 2020. During the initial discussions we determined that it would be good to examine the goals of previous meetings and determine the best path forward.

A mutual decision was made that to better create a roadmap; we needed to understand better who we were aiming to help. We decided that creating personas was very beneficial. Personas are fictional characters, which we are creating based upon our research to represent the different user types that might use Jenkins pipelines. Creating personas can help us step out of ourselves. It can help us to recognize that different people have different needs and expectations, and it can also help us to identify with the user we are building the roadmap for. Personas make the task at hand less complicated, they guide our ideation processes, and they can help us to achieve the goal of creating a good user experience for our target user group. A lot of that work can be found here: https://docs.google.com/document/d/1CdyzJwt50Wk3uUNsLMl2d4w2MGYss-phqet0s-KjbEs/edit The idea is to map the personas to a maturity model and then map the maturity model to the actual documentation. That maturity model can be found here: https://drive.google.com/file/d/1ByzWlPU0j1qM_gqspJppkNKkR5ZVLWlB/view

How Can I Get Involved

We have been meeting regularly to define personas to help us better create the SIG roadmap. We meet twice a week, once on Thursday for the EMEA timezone and once on Friday for the US timezone. Meeting notes can be found here: https://docs.google.com/document/d/1EhWoBplGl4M8bHz0uuP-iOynPGuONjcz4enQm8sDyUE/edit# and the calendar, if you would like to attend, is here: https://jenkins.io/event-calendar/. The previous recording of the meetings are located here: https://www.youtube.com/watch?v=pz_kPpb9C1w&list=PLN7ajX_VdyaOKKLBXek6iG8wTS24Ac7Y3

Next Steps

We have a lot of work to do and could use your help. If you would love to join us, check out the meeting link. If you would like to check out the personas and give feedback, also check out the link. Once we have wrapped up the personas work, we will start to identify the available documentation and ensure we have adequate documentation with the help of the Doc SIG. We will finally then start working to build out tools to help the community with pipelines in Jenkins better.

A Traditional Manager’s Guide for Modern Leadership Styles

There is no doubt that business organizations and educational institutions are now embracing the millennial evolution. Traditional leaders or those who were “schooled” from the top-down, military-like management style are beginning to understand that the needs, characteristics, and behaviors of the current and future workforce have changed significantly and therefore, they must adopt new leadership styles that would make them genuinely effective. That said, here are five modern leadership styles that every traditional manager must know.1. Innovative leadershipThis is the frontrunner in modern leadership styles that immensely surpassed the traditional, top-down, linear approach by thinking “outside of the box” and finding more than a thousand ways to achieve the organisation’s goals. Innovative leadership can be seen as the exact opposite of the conventional management style that was stuck in the idea that a once-proven-and-tested solution can lead to a one-size-fits-all formula when in reality, it was only a one-size-fits-one idea. Innovative leaders actively search for more approaches. To them, the goal is more important than the role. They do not easily dump things–even those that may appear like rubbish–but they are more than willing to roll up their sleeves, get their hands dirty along with their team, and take great ...


Read More on Datafloq

Friday, 28 February 2020

Securely Accessing Azure Data Sources from Azure Databricks

Azure Databricks is a Unified Data Analytics Platform that is a part of the Microsoft Azure Cloud. Built upon the foundations of Delta Lake, MLFlow , Koalas and Apache Spark, Azure Databricks is a first party service on Microsoft Azure cloud that provides one-click setup, native integrations with other Azure services, interactive workspace, and enterprise-grade security to power Data & AI use cases for small to large global customers. The platform enables true collaboration between different data personas in any enterprise, like Data Engineers, Data Scientists, Data Analysts and SecOps / Cloud Engineering.

In this blog which is first in a series of two, we’ll provide an overview of Azure Databricks architecture and how customers could connect to their own-managed instances of Azure data services in a secure manner.

Azure Databricks Architecture Overview

Azure Databricks is a managed application on Azure cloud. At a high-level, the architecture consists of a control / management plane and data plane. The control plane resides in a Microsoft-managed subscription and houses services such as web application, cluster manager, jobs service etc. In the default deployment, the data plane is a fully managed component in customer’s subscription that includes a VNET, NSG and a root storage account known as DBFS.

The data plane could also be deployed in a customer-managed VNET, to allow the SecOps and Cloud Engineering teams build security & network architecture for the service as per their enterprise governance policies. This capability is called Bring Your Own VNET or VNET Injection. The picture shows a representative view of such customer architecture.

Azure Databricks is a managed application on Azure cloud. At a high-level, the architecture consists of a control / management plane and data plane.

Secure connectivity to Azure Data Services

Enterprise Security is a core tenet of building software at both Databricks and Microsoft, and thus it’s considered as a first-class citizen in Azure Databricks. In the context of this blog, secure connectivity refers to ensuring that traffic from Azure Databricks to Azure data services remains on the Azure network backbone, with the inherent ability to whitelist Azure Databricks as an allowed source. As a security best practice, we recommend a couple of options which customers could use to establish such a data access mechanism to Azure Data services like Azure Blob Storage, Azure Data Lake Store Gen2, Azure Synapse Data Warehouse, Azure CosmosDB etc. Please read further for a discussion on Azure Private Link and Service Endpoints.

Option 1: Azure Private link

The most secure way to access Azure Data services from Azure Databricks is by configuring Private Link. As per Azure documentation – Private Link enables you to access Azure PaaS Services (for example, Azure Storage, Azure Cosmos DB, and SQL Database) and Azure hosted customer/partner services over a Private Endpoint in your virtual network. Traffic between your virtual network and the service traverses over the Microsoft network backbone, eliminating exposure from the public Internet. You can also create your own Private Link Service in your virtual network (VNet) and deliver it privately to your customers. The setup and consumption experience using Azure Private Link is consistent across Azure PaaS, customer-owned, and shared partner services. For details, please refer to this.

See below on how Azure Databricks and Private Link could be used together.

Azure Databricks and Azure Data Service Private Endpoints in separate VNETs

Azure Databricks and Azure Data Service Private Endpoints in separate VNETs

ALT TAG = Azure Databricks and Azure Data Service Private Endpoints in same VNET

Azure Databricks and Azure Data Service Private Endpoints in same VNET

Private Endpoint Considerations

Please consider the following before implementing the private endpoint:

  • Provides protection against data exfiltration by default. In the case of Azure Databricks, this would apply once customer whitelists access to specific services in the control plane.
  • Keeps traffic on Azure network backbone i.e public network is not used for any data flow.
  • Extends your private network address space to Azure Data services, i.e. the Azure data service effectively gets a private IP in one of your VNETs and could be treated as part of your larger private network.
  • Connect privately to Azure Data services in other regions i.e. VNET in region A could connect to endpoints in region B via Private Link.
  • Private Link is relatively bit more complex to set up as compared to other secure access mechanisms.
  • See the documentation for a detailed list of Private Link benefits and the service specific availability.

One example of where one could use Private Link is when a customer uses a few Azure Data services in production along with Azure Databricks, like Blob Storage, ADLS Gen2, SQL DB etc. The business would like the users to query the masked aggregated data from ADLS Gen2, but restrict them from making their way to the unmasked confidential data in other data sources. In that case, a private endpoint could be established only for ADLS Gen2 service using any of the sub-options discussed above.

This is how such an environment could be configured:

1 – Setup Private Link for ADLS Gen2

2 – Deploy Azure Databricks in your VNET

Please note that it’s possible to configure more than one Private Link per Azure Data service, which allows you to build an architecture that conforms to your enterprise governance needs.

Option 2: Azure Virtual Network Service Endpoints

As per Azure documentation, Virtual Network (VNET) service endpoints extend your virtual network private address space. The endpoints also extend the identity of your VNet to the Azure services over a direct connection. Endpoints allow you to secure your critical Azure service resources to only your virtual networks. Traffic from your VNet to the Azure service always remains on the Microsoft Azure network backbone.

Service endpoints provide the following benefits (source):

Improved security for your Azure service resources

Private address space for different virtual networks can overlap with each other. You can’t use overlapping network space to uniquely identify traffic that originates from a particular VNET. Once service endpoints are enabled for the subnets in your VNET, you can add a virtual network firewall rule to secure the Azure data services by extending your VNET identity to those resources. Such a configuration helps remove public access to those resources and allowing traffic only from your VNET.

Optimal routing for Azure data service traffic from your virtual network

Today, any routes on your VNET that are used to direct public network-headed traffic via your cloud/on-premises-based virtual appliances are also used for the Azure data service traffic. Service endpoints provide optimal routing for Azure traffic.

Keeping traffic on the Azure network backbone

Service endpoints always direct Azure data service traffic directly from your VNET to the resource on the Microsoft Azure network backbone. Keeping traffic on the Azure network backbone allows you to continue auditing and monitoring outbound Internet traffic from your virtual networks, through forced-tunneling, without impacting data service traffic. For more information about user-defined routes and forced-tunneling, see Azure virtual network traffic routing.

Simple to set up with no management overhead

You no longer need reserved, public IP addresses in your virtual networks to secure Azure data service resources through IP firewall. There are no Network Address Translation (NAT) or gateway devices required to set up the service endpoints. You can configure service endpoints through a simple setup for a subnet. There’s no additional overhead to maintaining the endpoints.

Azure Service Endpoint with Azure Databricks

Azure Service Endpoint with Azure Databricks

Azure Service Endpoint Considerations

Please consider the following before implementing the service endpoints:

  • Does not provide protection against data exfiltration by default.
  • Keeps traffic on Azure network backbone i.e public network is not used for any data flow.
  • Does not extend your private network address space to Azure Data services.
  • Cannot connect privately to Azure Data services in other regions (except for paired regions).
  • See the documentation for a detailed list of Azure Service Endpoint benefits and limitations.

Taking the same example as mentioned above for Private Link, and how it could look like with Service Endpoints. In this case, Azure Storage Service Endpoint could be configured on Azure Databricks subnets and the same subnets could then be whitelisted in ADLS Gen2 firewall rules.

This is how such an environment could be configured:

1 – Setup Service Endpoint for ADLS Gen2

2 – Deploy Azure Databricks in your VNET

3 – Configure IP firewall rules on ADLS Gen2

Getting Started with Secure Azure Data Access

We discussed a couple of options available to access Azure data services securely from your Azure Databricks environment. Based on your business specifics, you could either use Azure Private Link or Virtual Network Service Endpoints. Once the network connectivity approach is finalized, you could utilize secure auth approaches to connect to those resources:

In the next blog in this series, we’ll dive deep into how one could set up a buttoned-up locked down environment to prevent data exfiltration (in other words, implement a data loss prevention architecture). It would utilize a mix of the above discussed options and Azure Firewall. Please reach out to your Microsoft or Databricks account teams for any questions.

--

Try Databricks for free. Get started today.

The post Securely Accessing Azure Data Sources from Azure Databricks appeared first on Databricks.