Machine Archives - Business Intelligence Info

Tag Archives: Machine

IBM releases Qiskit modules that use quantum computers to improve machine learning

April 11, 2021 Big Data

Join GamesBeat Summit 2021 this April 28-29. Register for a free or VIP pass today.

IBM is releasing Qiskit Machine Learning, a set of new application modules that’s part of its open source quantum software. The new feature is the latest expansion of the company’s broader effort to get more developers to begin experimenting with quantum computers.

According to a blog post by the Qiskit Applications Team, the machine learning modules promise to help optimize machine learning by using quantum computers for some parts of the process.

“Quantum computation offers another potential avenue to increase the power of machine learning models, and the corresponding literature is growing at an incredible pace,” the team wrote. “Quantum machine learning (QML) proposes new types of models that leverage quantum computers’ unique capabilities to, for example, work in exponentially higher-dimensional feature spaces to improve the accuracy of models.”

Rather than replacing current computer architectures, IBM is betting that quantum computers will gain traction in the coming years by taking on very specific tasks that are offloaded from a classic computing system to a quantum platform. AI and machine learning are among the areas where IBM has said it’s hopeful that quantum can make an impact.

To make quantum more accessible, last year IBM introduced an open source quantum programming framework called Qiskit. The company has said it has the potential to speed up some applications by 100 times.

In the case of machine learning, the hope is that a system that offloads tasks to a quantum system could accelerate the training time. However, challenges remain, such as how to get large data sets in and out of the quantum machine without adding time that would cancel out any gains by the quantum calculations.

Developers who use Qiskit to improve their algorithms will have access to test them on IBM’s cloud-based quantum computing platform.

VentureBeat

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact.

Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

up-to-date information on the subjects of interest to you
our newsletters
gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
networking features, and more

Become a member

Let’s block ads! (Why?)

Big Data – VentureBeat

Announcing Plant-Based Machine Learning

April 5, 2021 BI News and Info

Around this time last year, we announced that we’d expanded our AI and ML capabilities to pets, truly making machine learning paw-ssible for anyone, regardless of whether you’re human or not.

And while we were proud of that effort, some of us wondered if we could go further. There are, after all, other groups of living creatures on this planet. Although we initially considered an expansion into the fungal kingdom, early experiments only resulted in a lot of moldy circuit boards.

But we didn’t let that stop us! We kept working at it, and this year, we’re excited to continue expanding the reach of AI and ML by providing members of the plant kingdom with the ability to run their own machine learning models, using the cloud-based infrastructure of RapidMiner Go. We were so pleased with how much we’ve expanded machine learning beyond humans that we went so far as to revise our understanding of our mission statement:

Our new toolset is based on a proprietary, state-of-the-art plant/computer interface called Lexical-Epicotyl Access for Plants and Herbs (LEAPH) that our green friends can use to access the Internet to find data and then use RapidMiner Go to build and train models.

Let’s take a look at a few of the plants that we worked with during prototyping to give you a sense of the kinds of models that plants might be interested in building and how they can use this technology to have a positive impact on our world.

Potato (Solanum tuberosum)

Potatoes get a bad rap—carbophobes hate them, they’re not infrequently forgotten in a bottom drawer where they turn into brown slush, and they’re even the butt of internet jokes, comparing them to crummy cameras and computers.

But with LEAPH, they can fight back against their haters! Using RapidMiner Go, our resident potato plants were able to build a model that could detect negative sentiment about potatoes on various websites and forums. Their next step is to build a model that can respond to try and discourage people from saying things like photos have “potato quality”.

So if you see anyone complaining about language that’s disparaging to potatoes online, you’ve probably run into the first potato-built sentiment analysis and influencer bot.

Avocado (Persea americana)

Avocados are superstars of the produce aisle—high in healthy fats, great on toast, and popular at parties in the form of guacamole.

But avocados have their problems, too. No avocado wants to be purchased only to be taken home and tossed in the trash when its all-too-short eating window comes and goes without anyone taking notice. This is especially concerning to the avocados since their unpredictable ripening times drive up costs for end consumers, potentially impacting their popularity and thus the number of avocados grown each year.

With our LEAPH interface, a grove of avocado trees was able to model the development of their fruits—including subjective measures like “how brown under the skin are they” and “do they have any of those weird thread things” that’s inaccessible to us humans on the outside—and try to create a model that lets people know when the avocados are ready to be eaten.

Unfortunately, initial testing didn’t result in a model that was able to accurately predict when an avocado was ripe with better than chance accuracy. However, the avocado trees are hopeful that they’ll eventually crack their own code, providing more predictability about their ripeness, and in turn, driving down costs so that restaurants can stop charging extra for guac.

Kale (Brassica oleracea)

Kale has exploded in popularity in the last decade or so, and we have no idea why—and neither does kale! With LEAPH and RapidMiner Go, kale was able to plumb the Internet to look for evidence and chart its rise to health-food hero. Although celebrity endorsements seem to have played a role, the main driver is that people love to talk about how health they are on social media. Since eating kale is seen as a key part of a healthy lifestyle, more social media has driven more kale sales.

And the kale ain’t complainin’!

Wrapping Up

These are just a few examples of the kinds of impacts that RapidMiner can have when it puts the power of AI and ML in anyone’s hands—even plants. We’re excited to see what other plants come up with as we roll out our LEAPH interfaces in the coming weeks.

Our new plant-based options are sure to rock the machine learning world to its roots, as is our new slogan: RapidMiner Go: literally so easy a potato can do it.

Let’s block ads! (Why?)

RapidMiner

Major flaws found in machine learning for COVID-19 diagnosis

March 24, 2021 Big Data

Join Transform 2021 for the most important themes in enterprise AI & Data. Learn more.

A coalition of AI researchers and health care professionals in fields like infectious disease, radiology, and ontology have found several common but serious shortcomings with machine learning made for COVID-19 diagnosis or prognosis.

After the start of the global pandemic, startups like DarwinAI, major companies like Nvidia, and groups like the American College of Radiology launched initiatives to detect COVID-19 from CT scans, X-rays, or other forms of medical imaging. The promise of such technology is that it could help health care professionals distinguish between pneumonia and COVID-19 or provide more options for patient diagnosis. Some models have even been developed to predict if a person will die or need a ventilator based on a CT scan. However, researchers say major changes are needed before this form of machine learning can be used in a clinical setting.

Researchers assessed more than 2,200 papers and, through a process of removing duplicates and irrelevant titles, narrowed results down to 320 papers that underwent a full text review for quality. Finally, 62 papers were deemed fit to be part of what authors refer to as a systematic review of published research and preprints shared on open research paper repositories like arXiv, bioRxiv, and medRxiv.

Of those 62 papers included in the analysis, roughly half made no attempt to perform external validation of training data, did not assess model sensitivity or robustness, and did not report the demographics of people represented in training data.

“Frankenstein” datasets, the kind made with duplicate images obtained from other datasets, were also found to be a common problem, and only one in five COVID-19 diagnosis or prognosis models shared their code so others can reproduce results claimed in literature.

“In their current reported form, none of the machine learning models included in this review are likely candidates for clinical translation for the diagnosis/prognosis of COVID-19,” the paper reads. “Despite the huge efforts of researchers to develop machine learning models for COVID-19 diagnosis and prognosis, we found methodological flaws and many biases throughout the literature, leading to highly optimistic reported performance.”

The research was published last week as part of the March issue of Nature Machine Intelligence by researchers from the University of Cambridge and University of Manchester. Other common issues they found with machine learning models developed using medical imaging data was virtually no assessment for bias and generally being trained without enough images. Nearly every paper reviewed was found to be at high or uncertain risk of bias; only six were considered at low risk of bias.

Publicly available datasets also commonly suffered from lower quality image formats and weren’t large enough to train reliable AI models. Researchers used the checklist for artificial intelligence in medical imaging (CLAIM) and radiomics quality score (RQS) to help assess the datasets and models.

“The urgency of the pandemic led to many studies using datasets that contain obvious biases or are not representative of the target population, for example, pediatric patients. Before evaluating a model, it is crucial that authors report the demographic statistics for their datasets, including age and sex distributions,” the paper reads. “Higher-quality datasets, manuscripts with sufficient documentation to be reproducible and external validation are required to increase the likelihood of models being taken forward and integrated into future clinical trials to establish independent technical and clinical validation as well as cost-effectiveness.”

Other recommendations suggested by the group of AI researchers and health care professionals include ensuring reproducibility of model performance results spelled out in research papers and considering how datasets are assembled and put together.

In other news at the intersection of COVID-19 and machine learning, earlier this week the Food and Drug Administration (FDA) approved emergency use authorization of a machine learning-based screening device which the agency says is the first approved in the U.S.

VentureBeat

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact.

Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

up-to-date information on the subjects of interest to you
our newsletters
gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
networking features, and more

Become a member

Let’s block ads! (Why?)

Big Data – VentureBeat

Not Getting the Most From Your Model Ops? Why Businesses Struggle With Data Science and Machine Learning

March 18, 2021 TIBCO Spotfire

TIBCO ModelOps scaled e1615222627944 696x366 Not Getting the Most From Your Model Ops? Why Businesses Struggle With Data Science and Machine Learning

Reading Time: 2 minutes

Companies have begun to recognize the value of integrating data science (DS) and machine learning (ML) across their organization to reap the benefits of the advanced analytics they can provide. As such, DS/ML has seen a surge in popularity and usage as businesses have invested heavily in this technology.

However, there’s a distinct difference between investing in DS/ML and managing to successfully gain tangible business value from that investment, and that’s where organizations are running into problems.

The Results Are in: Businesses Struggle With DS/ML Deployment Across the Board

We recently performed a global survey across 18 countries and 22 industries, including over a hundred business leaders and executives, more than half of which were in the C-Suite.

Of those respondents, just 14 percent reported that they are currently operationalizing DS/ML. Within that 14 percent, 24 percent can only use it in one functional area, far below the potential innovative capability of the technology.

Why are so few organizations able to follow through with model ops adoption? What are the barriers keeping businesses from operationalizing data science and machine learning?

The Devil’s in the Data

According to the survey results, while a lack of talented data scientists to build the models was listed in the top ten obstacles to DS/ML adoption, it was only cited by about 16 percent of respondents. On the other hand, seven of these ten, including the top four, were all data-related. Issues with data security, data privacy, data prep, and data access, in particular, were all cited by between 27 to 38 percent of respondents.

While there are many other issues to contend with, including lack of management and financial support and a clear integration strategy, security compliance and data privacy concerns are clearly a significant barrier when it comes to operationalizing DS/ML.

Why Overcoming These Problems are Critical for Innovation

Data scientists can develop as many models as they want for a business, but if they don’t get deployed, then they aren’t providing any value. For the modern digital business to have any hope of keeping up with the competition, model ops is a vital tool that can allow them to effectively operationalize DS/ML models, putting them into production and applying them to streaming, real-time data, edge applications, and more.

For a more in-depth breakdown of our survey results, you can check out our full ebook now. And if you’re ready to move past insights and into action, you can download our four-step guide to finding out what it takes to operationalize data science within your organization and get a leg-up on the competition.

Let’s block ads! (Why?)

The TIBCO Blog

OctoML raises $28M for machine learning deployment optimization

March 17, 2021 Big Data

Join Transform 2021 for the most important themes in enterprise AI & Data. Learn more.

It’s well known that most businesses face challenges deploying AI in production, and that’s led to the rise of markets serve those needs. In the latest news of advance for such a company, OctoML today raised a $ 28 million Series B funding round.

OctoML helps businesses accelerate and deploy AI and relies on the open-source technology Apache TVM machine learning compiler framework. The funding will be used for OctoML to continue building out products like the Octomizer platform and invest in the company’s go-to-market strategy and customer service teams.

“We started the TVM work as a research project at the University of Washington about five years ago, and all the key people in the project are part of, they all got their PhDs and are part of the company now,” OctoML CEO and cofounder Luis Ceze told VentureBeat. “We’re focused on making inference fast on any hardware, and support cloud and edge deployments.”

Last month, OctoML joined more than 20 startups who have banded together to create the AI Infrastructure Alliance, an effort involving startups like Algorithmia and Determined AI for interoperability between the offerings from AI startups and advance alternatives to popular cloud AI services.

The $ 28 million funding was led by Addition Capital led the round with participation from existing investors Madrona Venture Group and Amplify Partners.

OctoML has raised $ 47 million to date. A $ 3.9 million seed funding round was held in October 2019.

VentureBeat

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact.

Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

up-to-date information on the subjects of interest to you
our newsletters
gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
networking features, and more

Become a member

Let’s block ads! (Why?)

Big Data – VentureBeat

Why Choose RapidMiner for Your Data Science & Machine Learning Software?

March 6, 2021 BI News and Info

Recent recognition received from Gartner and RapidMiner’s end-users

If you follow RapidMiner’s blog, you know that we shy away from self-promotion. We’re much more interested in lending expertise that can help your data science efforts—model creation best practices, cutting-edge industry examples of AI/ML usage, and recommendations on how to overcome common business challenges that can prevent operationalization of machine learning.

However, we’ll make exceptions when our product team gets some much-deserved recognition. Their work empowers people to leverage machine learning more effectively, regardless of their role or technical skill-level. They make it easier to solve business problems and tackle new challenges through a more informed, analytical lens.

So, with that said…

RapidMiner Named a Visionary in this year’s Magic Quadrant for Data Science & Machine Learning Platforms

We’re excited to share that Gartner has named RapidMiner Visionary in this year’s Magic Quadrant for Data Science & Machine Learning Platforms! If you’re unfamiliar with Gartner Magic Quadrants, they offer an in-depth look at markets, the trends that define them, and key participants. Put simply, Magic Quadrant help companies get a sense for vendors’ strengths and weaknesses when they’re evaluating a purchase. By providing actionable insights and advice, they ensure that readers are making the most informed buying decisions they can.

We believe this year’s Magic Quadrant builds on the positive trends that we saw in 2020’s Gartner Peer Insights for Machine Learning and Data Science, which is the firm’s ranking of platforms in the market based on real end-user reviews. The feedback from RapidMiner users was so positive that we were awarded a Customers’ Choice Award, signifying that customers rated our platform strongly relative to competitors in the market. We’ve seen similar feedback from other reviewers on Forrester & G2 Crowd.

Over the past year, we’ve been focused on expanding our platform to meet the needs of larger enterprises. An examination of the challenges that these enterprises face when trying to use AI led us to invest in key areas like collaboration, governance, and explainable AI. The recent recognition we’ve received from both Gartner’s analysts & RapidMiner’s end-users validates our investment in those areas.

What Does it Mean to be Visionary?

The reason that we view this as such a key milestone isn’t just because we were recognized, but it’s because of the advancements that we’ve made in the realms of multi-persona collaboration, explainable AI, and model governance.

Multi-persona collaboration

In terms of collaboration, it’s always been a goal of ours to allow data scientists and business experts to work together to solve business problems because, while data scientists bring a wealth of knowledge to the table when it comes to building models and extracting insights, they don’t have experience working in functional areas every day, so they may not have as much context about business problems. By contrast, a Head of Production is in tune with the fact that they need to lower product defects & improve yield; they just don’t have the coding background to build models than can give them insight on where to start.

By allowing these two groups to work together in a single platform, companies using RapidMiner can ensure that they’re addressing the right problems and building technically sound models that will have strong business impact.

Explainable AI

In addition to getting teams to work better together, we’ve also been focused on ways to help you visualize and communicate the results of your data science work to others. We’re well aware of the fact that if can’t easily explain what your model is doing and how, it’s unlikely that you’re going to be able to get it implemented to have real business impact. And what better way to present model insights than with visualization?

By enabling users to visualize and explain the models that they’ve built, whether they’re in development or production, RapidMiner creates greater transparency, gives users full control over insights, and helps ensure that models can make it across the finish line and return dividends on the investment in machine learning.

Model governance

Lastly, we’ve invested a lot into helping companies establish secure, governable data science practices. This is rooted in the belief that no enterprise AI initiative is worth investing in if you can’t guarantee that your data is safe and properly governed. That’s why we’ve implemented auditable project-tracking, Single Sign-On (SSO), and strong identity and access management (IAM) capabilities within our platform, allowing admins to secure their AI pipeline, all in one place.

Looking Ahead

In addition to the three main points above, the Magic Quadrant also notes RapidMiner’s clear vision for what features need to be implemented in the future, as well as our ability to get those features right. For example, in 2020, we notably added capabilities that “enable users to perform automated feature engineering, and share and store features across an organization, thus enhancing reusability and reproducibility.”

Again, while the posts on this blog aren’t typically about RapidMiner, we’re proud to be recognized for our commitment to reinvent enterprise AI so that anyone has the power to positively shape the future. Check out everything Gartner has to say by reading the full report.

Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, express or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

Gartner Magic Quadrant for Data Science and Machine Learning Platforms, Peter Krensky, Carlie Idoine, Erick Brethenoux, Pieter den Hamer, Farhan Choudhary, Afraz Jaffri, Shubhangi Vashisth,1^st March 2021.

Let’s block ads! (Why?)

RapidMiner

Why machine learning strategies fail

February 26, 2021 Big Data

Most companies are struggling to develop working artificial intelligence strategies, according to a new survey by cloud services provider Rackspace Technology. The survey, which includes 1,870 organizations in a variety of industries, including manufacturing, finance, retail, government, and healthcare, shows that only 20 percent of companies have mature AI/machine learning initiatives. The rest are still trying to figure out how to make it work.

There’s no questioning the promises of machine learning in nearly every sector. Lower costs, improved precision, better customer experience, and new features are some of the benefits of applying machine learning models to real-world applications. But machine learning is not a magic wand. And as many organizations and companies are learning, before you can apply the power of machine learning to your business and operations, you must overcome several barriers.

Three key challenges companies face when integrating AI technologies into their operations are in the areas of skills, data, and strategy, and Rackspace’s survey paints a clear picture of why most machine learning strategies fail.

Machine learning is about data

Machine learning models live on compute resources and data. Thanks to a variety of cloud computing platforms, access to the hardware needed to train and run AI models has become much more accessible and affordable.

But data continues to remain a major hurdle in different stages of planning and adopting an AI strategy. Thirty-four percent of the respondents in the Rackspace survey stated poor data quality as the main reason for the failure of machine learning research and development, and another 31 percent said they lacked production-ready data.

This highlights one of the main hurdles when applying machine learning techniques to real-world problems. While the AI research community has access to many public datasets for training and testing their latest machine learning technologies, when it comes to applying those technologies to real applications, getting access to quality data is not easy. This is especially true in industrial, health, and government sectors, where data is often scarce or subject to strict regulations.

Data problems crop up again when machine learning initiatives move from the research to the production phase. Data quality remains the top barrier when it comes to using machine learning to extract valuable insights. Data engineering problems also pose a significant problem, such as data being siloed, lack of talent to connect disparate data sources, and not being fast enough to process data in a meaningful way.

Above: Data accounts for most key problems in gaining actionable insights from machine learning models (Source: Rackspace Technology)

Both startups and established companies suffer from data problems, though scale seems to be the key differentiator between the two, according to Jeff DeVerter, CTO of Rackspace Technology. “Startups tend to be constrained with not all the right resources to implement a quality data pipeline and consistently managing it over time,” DeVerter said to TechTalks in written comments. “Enterprises usually have scale on their side and with that comes the rigor that’s required.”

The best way companies can prepare for the data challenges of AI strategies is to do a full evaluation of their data infrastructure. Eliminating silos should be a key priority in every machine learning initiative. Companies should also have the right procedures for cleaning their data to improve the accuracy and performance of their machine learning models.

AI talent is still in high demand

The second area of struggle for most companies is access to machine learning and data science talent. According to Rackspace’s survey, lack of in-house expertise was the second biggest driver of failure in machine learning R&D initiatives. Lack of skill and difficulty in hiring was also a key barrier in adopting AI technologies.

Above: Many companies struggle with acquiring the talent to implement their AI strategies (Source: Rackspace Technology)

With machine learning and deep learning having reached mainstream use in production environments only recently, many smaller companies don’t have data scientists and machine learning engineers who can develop AI models.

And the average salary of data scientists and machine learning engineers matches those of experienced software engineers, which makes it difficult for many companies to put together a talented team that can lead its AI initiative.

While the shortage of machine learning and data science talent is well known, one thing that has gone mostly unnoticed is the need for more data engineers, the people who set up, maintain, and update databases, data warehouses, and data lakes. Per Rackspace’s figures, many initiatives fail because companies don’t have the talent to adapt their data infrastructure for machine learning purposes. Breaking down silos, migrating to cloud, setting up Hadoop clusters, and creating hybrid systems that can leverage the power of different platforms are some areas where companies are sorely lacking. And these shortcomings prevent them from making company-wide deployments of machine learning initiatives.

With the development of new machine learning and data science tools, the talent problem has become less intense. Google, Microsoft, and Amazon have launched platforms that make it easier to develop machine learning models. An example is Microsoft’s Azure Machine Learning service, which provides a visual interface with drag-and-drop components and makes it easier to create ML models without coding. Another example is Google’s AutoML, which automates the tedious process of hyperparameter tuning. While these tools are not a replacement for machine learning talent, they lower the barrier for people who want to enter the field and will enable many companies to reskill their tech talent for these growing fields.

“Lack of in-house data science talent is not the barrier it once was now that more of these services are able to use their own ML to help in this regard as well consulting firms having these talents on-staff,” DeVerter said.

Other developments in the field are the evolution of cloud storage and analysis platforms, which have considerably reduced the complexity of creating the seamless data infrastructures needed to create and run AI systems. An example is Google’s BigQuery, a cloud-based data warehouse that can run queries across vast amounts of data stored in various sources with minimal effort.

We’re also seeing growing compatibility and integration capabilities in machine learning tools, which will make it much easier for organizations to integrate ML tools into their existing software and data ecosystem.

Before entering an AI initiative, every organization must make a full evaluation of in-house talent, available tools, and integration possibilities. Knowing how much you can rely on your own engineers and how much it will cost you to hire talent will be a defining factor in the success or failure of your machine learning initiatives. Also, consider whether re-skilling is a possible course of action. If you can upskill your engineers to take on data science and machine learning projects, you will be better off in the long run.

Outsourcing AI talent

Another trend that has seen growth in recent years is the outsourcing of AI initiatives. Only 38 percent of the Rackspace survey respondents relied on in-house talent to develop AI applications. The rest were either fully outsourcing their AI projects or employing a combination of in-house and outsourced talent.

Above: Most companies rely on outside talent to plan and implement their AI initiatives (Source: Rackspace Technology)

There are now several companies that specialize in developing and implementing AI strategies. An example is C3.ai, an AI solutions provider that specializes in several industries. C3.ai provides AI tools on top of existing cloud providers such as Amazon, Microsoft, and Google. The company also provides AI consultancy and expertise to take customers step by step through the strategizing and implementation phases.

According to the Rackspace report: “A mature provider can bring everything from strategy to implementation to maintenance and support over time. Strategy can sidestep the areas where AI and machine learning efforts may lose momentum or get lost in complexity. Hands-on experts can also spare organizations from the messy work of cleanup and maintenance. Such expertise, taken together, can make all the difference in finally achieving success.”

It is worth noting, however, that fully turning over an organization’s AI strategy to outside providers can be a double-edged sword. A successful strategy requires close cooperation between AI specialists and subject matter experts from the company that is implementing the strategy.

“This is very similar to companies who move to a DevOps development methodology and attempt to outsource the entirety of the development. DevOps requires a close partnership between the developers, business analysts, and others in the business,” DeVerter said. “In the same way, AI projects require strategy and technical expertise — but also require a tight partnership with the business as well as leadership.”

Outsourcing AI talent must be done meticulously. While it can speed up the process of developing and implementing an AI strategy, you must make sure that your experts are fully involved in the process. Ideally, you should be able to develop your own in-house team of data scientists and machine learning engineers as you work with outside experts.

How do you evaluate your AI strategy?

Finally, another area that is causing much pain for companies embarking on an AI journey is forecasting the outcome and value of AI strategies. Given the application of machine learning being new to many areas, it’s hard to know in advance how long an AI strategy will take to plan and implement and what the return on investment will be. This in turn makes it difficult for innovators in organizations to get others on board when it comes to garnering support for AI initiatives.

Of the respondents of the Rackspace survey, 18 percent believed that a lack of clear business case was the main barrier to adopting AI strategies. Lack of commitment from executives was also among the top barriers. Lack of use cases and commitment from senior management show up again among the top challenges in the machine learning journey.

“AI often wanders around as a solution looking for a problem within organizations. I believe this is one of the greatest impediments to its wide-scale adoption within organizations,” DeVerter said. “As AI practitioners can demonstrate practical examples of how AI can benefit their specific company — leadership will further fund those activities. Like any business venture — leadership needs to know how it will either help them save or make money.”

Evaluating the outcome of AI initiatives is very difficult. According to the survey, the top-two key performance indicators (KPI) for measuring the success of AI initiatives were profit margins and revenue growth. Understandably, this focus on quick profits is partly due to the high costs of AI initiatives. According to the Rackspace survey, organizations spend a yearly average of $ 1.06 million on AI initiatives.

But while a good AI initiative should result in revenue growth and lower costs, in many cases, the long-term value of machine learning is the development of new use cases and products.

“Short-term financial gains can be myopic if they aren’t paired with a long-term strategy that can be funded by those short-term gains,” DeVerter said.

If you’re in charge of the AI initiative in your organization, make sure to clearly lay out the use cases, the costs, and the benefits of your AI strategy. Decision-makers should have a clear picture of what their company will be embarking on. They should understand the short-term benefits of investing in AI, but they should also know what they will gain in the long run.

Ben Dickson is a software engineer and the founder of TechTalks. He writes about technology, business, and politics. This post was originally published here as a series exploring the business of artificial intelligence.

VentureBeat

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact.

Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

up-to-date information on the subjects of interest to you
our newsletters
gated thought-leader content and discounted access to our prized events, such as Transform
networking features, and more

Become a member

Let’s block ads! (Why?)

Big Data – VentureBeat

You don’t code? Do machine learning straight from Microsoft Excel

December 31, 2020 Big Data

Machine learning and deep learning have become an important part of many applications we use every day. There are few domains that the fast expansion of machine learning hasn’t touched. Many businesses have thrived by developing the right strategy to integrate machine learning algorithms into their operations and processes. Others have lost ground to competitors after ignoring the undeniable advances in artificial intelligence.

But mastering machine learning is a difficult process. You need to start with a solid knowledge of linear algebra and calculus, master a programming language such as Python, and become proficient with data science and machine learning libraries such as Numpy, Scikit-learn, TensorFlow, and PyTorch.

And if you want to create machine learning systems that integrate and scale, you’ll have to learn cloud platforms such as Amazon AWS, Microsoft Azure, and Google Cloud.

Naturally, not everyone needs to become a machine learning engineer. But almost everyone who is running a business or organization that systematically collects and processes can benefit from some knowledge of data science and machine learning. Fortunately, there are several courses that provide a high-level overview of machine learning and deep learning without going too deep into math and coding.

But in my experience, a good understanding of data science and machine learning requires some hands-on experience with algorithms. In this regard, a very valuable and often-overlooked tool is Microsoft Excel.

To most people, MS Excel is a spreadsheet application that stores data in tabular format and performs very basic mathematical operations. But in reality, Excel is a powerful computation tool that can solve complicated problems. Excel also has many features that allow you to create machine learning models directly into your workbooks.

While I’ve been using Excel’s mathematical tools for years, I didn’t come to appreciate its use for learning and applying data science and machine learning until I picked up Learn Data Mining Through Excel: A Step-by-Step Approach for Understanding Machine Learning Methods by Hong Zhou.

Learn Data Mining Through Excel takes you through the basics of machine learning step by step and shows how you can implement many algorithms using basic Excel functions and a few of the application’s advanced tools.

While Excel will in no way replace Python machine learning, it is a great window to learn the basics of AI and solve many basic problems without writing a line of code.

Linear regression machine learning with Excel

Linear regression is a simple machine learning algorithm that has many uses for analyzing data and predicting outcomes. Linear regression is especially useful when your data is neatly arranged in tabular format. Excel has several features that enable you to create regression models from tabular data in your spreadsheets.

One of the most intuitive is the data chart tool, which is a powerful data visualization feature. For instance, the scatter plot chart displays the values of your data on a cartesian plane. But in addition to showing the distribution of your data, Excel’s chart tool can create a machine learning model that can predict the changes in the values of your data. The feature, called Trendline, creates a regression model from your data. You can set the trendline to one of several regression algorithms, including linear, polynomial, logarithmic, and exponential. You can also configure the chart to display the parameters of your machine learning model, which you can use to predict the outcome of new observations.

You can add several trendlines to the same chart. This makes it easy to quickly test and compare the performance of different machine learning models on your data.

You don’t code? Do machine learning straight from Microsoft Excel

Above: Excel’s Trendline feature can create regression models from your data.

In addition to exploring the chart tool, Learn Data Mining Through Excel takes you through several other procedures that can help develop more advanced regression models. These include formulas such as LINEST and LINREG, which calculate the parameters of your machine learning models based on your training data.

The author also takes you through the step-by-step creation of linear regression models using Excel’s basic formulas such as SUM and SUMPRODUCT. This is a recurring theme in the book: You’ll see the mathematical formula of a machine learning model, learn the basic reasoning behind it, and create it step by step by combining values and formulas in several cells and cell arrays.

While this might not be the most efficient way to do production-level data science work, it is certainly a very good way to learn the workings of machine learning algorithms.

Other machine learning algorithms with Excel

Beyond regression models, you can use Excel for other machine learning algorithms. Learn Data Mining Through Excel provides a rich roster of supervised and unsupervised machine learning algorithms, including k-means clustering, k-nearest neighbor, naive Bayes classification, and decision trees.

The process can get a bit convoluted at times, but if you stay on track, the logic will easily fall in place. For instance, in the k-means clustering chapter, you’ll get to use a vast array of Excel formulas and features (INDEX, IF, AVERAGEIF, ADDRESS, and many others) across several worksheets to calculate cluster centers and refine them. This is not a very efficient way to do clustering, but you’ll be able to track and study your clusters as they become refined in every consecutive sheet. From an educational standpoint, the experience is very different from programming books where you provide a machine learning library function your data points and it outputs the clusters and their properties.

Above: When doing k-means clustering on Excel, you can follow the refinement of your clusters on consecutive sheets.

In the decision tree chapter, you will go through the process calculating entropy and selecting features for each branch of your machine learning model. Again, the process is slow and manual, but seeing under the hood of the machine learning algorithm is a rewarding experience.

In many of the book’s chapters, you’ll use the Solver tool to minimize your loss function. This is where you’ll see the limits of Excel, because even a simple model with a dozen parameters can slow your computer down to a crawl, especially if your data sample is several hundred rows in size. But the Solver is an especially powerful tool when you want to fine-tune the parameters of your machine learning model.

Above: Excel’s Solver tool fine-tunes the parameters of your model and minimizes loss functions.

Deep learning and natural language processing with Excel

Learn Data Mining Through Excel shows that Excel can even express advanced machine learning algorithms. There’s a chapter that delves into the meticulous creation of deep learning models. First, you’ll create a single layer artificial neural network with less than a dozen parameters. Then you’ll expand on the concept to create a deep learning model with hidden layers. The computation is very slow and inefficient, but it works, and the components are the same: cell values, formulas, and the powerful Solver tool.

Above: Deep learning with Microsoft Excel gives you a view under the hood of how deep neural networks operate.

In the last chapter, you’ll create a rudimentary natural language processing (NLP) application, using Excel to create a sentiment analysis machine learning model. You’ll use formulas to create a “bag of words” model, preprocess and tokenize hotel reviews, and classify them based on the density of positive and negative keywords. In the process you’ll learn quite a bit about how contemporary AI deals with language and how much different it is from how we humans process written and spoken language.

Excel as a machine learning tool

Whether you’re making C-level decisions at your company, working in human resources, or managing supply chains and manufacturing facilities, a basic knowledge of machine learning will be important if you will be working with data scientists and AI people. Likewise, if you’re a reporter covering AI news or a PR agency working on behalf of a company that uses machine learning, writing about the technology without knowing how it works is a bad idea (I will write a separate post about the many awful AI pitches I receive every day). In my opinion, Learn Data Mining Through Excel is a smooth and quick read that will help you gain that important knowledge.

Beyond learning the basics, Excel can be a powerful addition to your repertoire of machine learning tools. While it’s not good for dealing with big data sets and complicated algorithms, it can help with the visualization and analysis of smaller batches of data. The results you obtain from a quick Excel mining can provide pertinent insights in choosing the right direction and machine learning algorithm to tackle the problem at hand.

Ben Dickson is a software engineer and the founder of TechTalks. He writes about technology, business, and politics.

VentureBeat

VentureBeat’s mission is to be a digital townsquare for technical decision makers to gain knowledge about transformative technology and transact.

Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

up-to-date information on the subjects of interest to you,
our newsletters
gated thought-leader content and discounted access to our prized events, such as Transform
networking features, and more.

Become a member

Let’s block ads! (Why?)

Big Data – VentureBeat

OcéanIA treats climate change like a machine learning grand challenge

December 9, 2020 Big Data

Self-driving cars. Artificial general intelligence. Beating a human in a game of chess. Grand challenges are tasks that can seem like moonshots that, if achieved, will move the entire machine learning discipline forward. Now a team of researchers with the recently established OcéanIA is treating the study of the ocean and climate change as a machine learning grand challenge. The four-year project that brings together more than a dozen AI researchers and scientists shared some initial plans this week.

The OcéanIA project begins with a focus on the automated recognition of plankton species, many of which have not been documented. Next to trees and forests, plankton and the processes they’re a part of in the ocean are some of the largest carbon-capturing methods on Earth. Last year, the Intergovernmental Panel on Climate Change identified a correlation between climate change and the ocean’s ability to sequester carbon, produce oxygen, and support biodiversity. A study released in May found that plankton absorb twice as much carbon as scientists previously thought. A team of about 15 researchers are working on OcéanIA across machine learning and fields like biology, said Inria Chile Research Center director Nayat Sánchez-Pi.

“These crucial ecological services provided by plankton need to be better measured, monitored, and protected in order to maintain the ocean’s stability, to mitigate the various effects of climate change, and ensure the food security of population,” Sánchez-Pi said. “Oceans today we can say are the last unknown, and understanding the role of oceans in climate change is not only important but also a challenge for modern AI and applied ML.”

Sánchez-Pi was one of four keynote speakers at the Latinx in AI workshop Monday as part of the Neural Information Processing Systems (NeurIPS) conference. Affinity workshops at the conference include Black in AI, Jews in AI, Queer in AI, and Women in Machine Learning. For the first time this year, NeurIPS will host Indigenous in AI and Muslims in AI workshops.

Luis Martí and Sánchez-Pi are also lead authors of a paper detailing OcéanIA that was accepted for publication at the Tackling Climate Change workshop being held Friday, the first published work associated with the project. More than 90 research and proposal papers were accepted for publication at the climate change workshop.

Machine learning challenges presented by the need to study plankton and oceans range from working with small datasets and few-shot learning methods to transfer learning, the process of repurposing a model for new tasks.

Unsupervised and semi-supervised methods will be used to identify particular plankton species. There are an estimated 70,000 unknown plankton species in the ocean today. Explainability will be used to tell the difference between different species.

Specific challenges listed in the proposal paper include the creation of models that incorporate complex knowledge about plankton into ocean-climate models and the development of “a metabolic model including the main microbial oceanic compartments and couple it with physics,” as well as computer vision for identifying plankton from satellite images. Satellite imagery is a traditional method researchers use to understand plankton populations.

At the previous Tackling Climate Change workshop at NeurIPS, researchers like Google Brain cofounder Andrew Ng argued that making scientific progress toward solving climate change and progress toward machine learning grand challenges is a two-way street.

“I do think [for] the future of AI and ML, a great challenge is scientific discovery. Indeed, how to embed prior knowledge, scientific reasoning, and how to be able to deal with small data,” Institute for Computational Sustainability director Carla Gomes said during a panel discussion one year ago.

Last year at NeurIPS, Facebook chief AI scientist Yann LeCun talked about energy efficiency as another worthy challenge for AI researchers.

OcéanIA treats climate change like a machine learning grand challenge

Above: Tara sampling method

Data to study plankton species will come courtesy of Tara Océans Foundation, which has undertaken 11 expeditions since 2005. The 12th Tara Océans expedition will focus on the study of the ocean ecosystem. It begins this month and continues through July 2022. The expedition will travel along the coast of Africa, Europe, and South America. Along the way, participants will collect samples at depths ranging from the surface of the sea to 1,000 meters deep.

More than 35 scientific institutions from the University of Sao Paolo in Brazil to the University of Cape Town in South Africa will participate in the study of samples and data collected by Tara Océans. An upcoming leg of the expedition will go through the Patagonia region of Chile.

Let’s block ads! (Why?)

Big Data – VentureBeat

AI Weekly: The state of machine learning in 2020

November 27, 2020 Big Data

It’s hard to believe, but a year in which the unprecedented seemed to happen every day is just weeks from being over. In AI circles, the end of the calendar year means the rollout of annual reports aimed at defining progress, impact, and areas for improvement.

The AI Index is due out in the coming weeks, as is CB Insights’ assessment of global AI startup activity, but two reports — both called The State of AI — have already been released.

Last week, McKinsey released its global survey on the state of AI, a report now in its third year. Interviews with executives and a survey of business respondents found a potential widening of the gap between businesses that apply AI and those that do not.

The survey reports that AI adoption is more common in tech and telecommunications than in other industries, followed by automotive and manufacturing. More than two-thirds of respondents with such use cases say adoption increased revenue, but fewer than 25% saw significant bottom-line impact.

Along with questions about AI adoption and implementation, the McKinsey State of AI report examines companies whose AI applications led to EBIT growth of 20% or more in 2019. Among the report’s findings: Respondents from those companies were more likely to rate C-suite executives as very effective, and the companies were more likely to employ data scientists than other businesses were.

At rates of difference of 20% to 30% or more compared to others, high-performing companies were also more likely to have a strategic vision and AI initiative road map, use frameworks for AI model deployment, or use synthetic data when they encountered an insufficient amount of real-world data. These results seem consistent with a Microsoft-funded Altimeter Group survey conducted in early 2019 that found half of high-growth businesses planned to implement AI in the year ahead.

If there was anything surprising in the report, it’s that only 16% of respondents said their companies have moved deep learning projects beyond a pilot stage. (This is the first year McKinsey asked about deep learning deployments.)

Also surprising: The report showed that businesses made little progress toward mounting a response to risks associated with AI deployment. Compared with responses submitted last year, companies taking steps to mitigate such risks saw an average 3% increase in response to 10 different kinds of risk — from national security and physical safety to regulatory compliance and fairness. Cybersecurity was the only risk that a majority of respondents said their companies are working to address. The percentage of those surveyed who consider AI risks relevant to their company actually dropped in a number of categories, including in the area of equity and fairness, which declined from 26% in 2019 to 24% in 2020.

McKinsey partner Roger Burkhardt called the survey’s risk results concerning.

“While some risks, such as physical safety, apply to only particular industries, it’s difficult to understand why universal risks aren’t recognized by a much higher proportion of respondents,” he said in the report. “It’s particularly surprising to see little improvement in the recognition and mitigation of this risk, given the attention to racial bias and other examples of discriminatory treatment, such as age-based targeting in job advertisements on social media.”

Less surprising, the survey found an uptick in automation in some industries during the pandemic. VentureBeat reporters have found this to be true across industries like agriculture, construction, meatpacking, and shipping.

“Most respondents at high performers say their organizations have increased investment in AI in each major business function in response to the pandemic, while less than 30% of other respondents say the same,” the report reads.

The McKinsey State of AI in 2020 global survey was conducted online from June 9 to June 19 and garnered nearly 2,400 responses, with 48% reporting that their companies use some form of AI. A 2019 McKinsey survey of roughly the same number of business leaders found that while nearly two-thirds of companies reported revenue increases due to the use of AI, many still struggled to scale its use.

The other State of AI

A month before McKinsey published its business survey, Air Street Capital released its State of AI report, which is now in its third year. The London-based venture capital firm found the AI industry to be strong when it comes to company funding rounds, but its report calls centralization of AI talent and compute “a huge problem.” Other serious problems Air Street Capital identified include ongoing brain drain from academia to industry and issues with reproducibility of models created by private companies.

A number of the report’s conclusions are in line with a recent analysis of AI research papers that found the concentration of deep learning activity among Big Tech companies, industry leaders, and elite universities is increasing inequality. The team behind this analysis says a growing “compute divide” could be addressed in part by the implementation of a national research cloud.

As we inch toward the end of the year, we can expect more reports on the state of machine learning. The state of AI reports released in the past two months demonstrate a variety of challenges but suggest AI can help businesses save money, generate revenue, and follow proven best practices for success. At the same time, researchers are identifying big opportunities to address the various risks associated with deploying AI.

For AI coverage, send news tips to Khari Johnson and Kyle Wiggers and AI editor Seth Colaner — and be sure to subscribe to the AI Weekly newsletter and bookmark our AI Channel.

Thanks for reading,

Khari Johnson

Senior AI Staff Writer

Let’s block ads! (Why?)

Big Data – VentureBeat

Tag Archives: Machine

VentureBeat

Potato (Solanum tuberosum)

Avocado (Persea americana)

Kale (Brassica oleracea)

Wrapping Up

VentureBeat

The Results Are in: Businesses Struggle With DS/ML Deployment Across the Board

The Devil’s in the Data

Why Overcoming These Problems are Critical for Innovation

VentureBeat

Recent recognition received from Gartner and RapidMiner’s end-users

RapidMiner Named a Visionary in this year’s Magic Quadrant for Data Science & Machine Learning Platforms

What Does it Mean to be Visionary?

Multi-persona collaboration

Explainable AI

Model governance

Looking Ahead

Machine learning is about data

AI talent is still in high demand

Outsourcing AI talent

How do you evaluate your AI strategy?

VentureBeat

Linear regression machine learning with Excel

Other machine learning algorithms with Excel

Deep learning and natural language processing with Excel

Excel as a machine learning tool

VentureBeat

The other State of AI

Recent Posts

Categories

Archives