finding Archives - Business Intelligence Info

Tag Archives: finding

Finding a monotonic polynomial over an interval

January 2, 2020 BI News and Info

I want to find a polynomial of specified degree $ d$ defining the function $ f:[0,1]\to[0,1]$ satisfying $ f(0)=0,f(1)=1$ and $ f$ is monotonically increasing over the interval. This seems like an easy enough task especially since there’s the trivial solution $ f(x)=x^d$ , and although my code seems to work decently well for $ d=2,3$ , it takes around a minute on my machine to produce an answer for $ d=4$ and it couldn’t get anything in the time I set it running for $ d=5$ . I’m looking to test this for values of $ d$ up to $ 10^3$ and find multiple instances for $ f$ , so this is certainly not going to work. What am I doing wrong and how can I make this efficient?

Here is my code:

deg = 4;
f[x_] := Sum[Subscript[c, k] x^k, {k, 1, deg}];
FindInstance[{
   f[1] == 1,
   ForAll[x, 0 <= x <= 1, f'[x] >= 0]
   }, Table[Subscript[c, i], {i, 1, deg}], Reals]

Let’s block ads! (Why?)

Recent Questions – Mathematica Stack Exchange

A super-fast machine learning model for finding user search intent

November 30, 2019 Big Data

In April 2019, Benjamin Burkholder (who is awesome, by the way) published a Medium article showing off a script he wrote that uses SERP result features to infer a user’s search intent. The script uses the SerpAPI.com API for its data and labels search queries in the following way:

Informational — The person is looking for more information on a topic. This is indicated by whether an answer box or PAA (people also ask) boxes are present.
Navigational — The person is searching for a specific website. This is indicated by whether a knowledge graph is present or if site links are present.
Transactional — The person is aiming to purchase something. This is indicated by whether shopping ads are present.
Commercial Investigation — The person is aiming to make a purchase soon but is still researching. This is indicated by whether paid ads are present, an answer box is present, PAAs are present, or if there are ads present at the bottom of the SERP.

This is one of the coolest ways to estimate search intent, because it uses Google’s understanding of search intent (as expressed by the SERP features shown for that search).

The one problem with Burkholder’s approach is its reliance on the Serp API. If you have a large set of search queries you want to find intent for, you need to pass each query phrase through the API, which then actually does the search and returns the SERP feature results, which Burkholder’s script can then classify. So on a large set of search queries, this is time consuming and prohibitively expensive.

SerpAPI charges ~$ 0.01 per keyword, so analyzing 5,000 keywords will cost you $ 50. Running these results through Burkholder’s labeler script also takes 3 to 5 hours to get through these 5,000 keywords.

So I got to thinking: What if I adapted Burkholder’s approach so that, rather than use it to classify intent directly, I could use it to train a machine learning model that I would then use to classify intent? In other words, I’d incur one-time costs to produce my Burkholder-labeled training set, and, assuming it was accurate enough, I could then use that training set for all further classification, cost free.

With an accurate training set, anyone could label huge numbers of keywords super quickly, without spending a dime.

Finding a model

Hamlet Batista has written a few stellar posts about how to leverage Natural Language models like BERT for labeling intent.

In his posts, he uses an existing intent labeling model that returns categories from Kaggle’s Question Answering Dataset. While these labels can be useful, they are not really “intent categories” in line with what we typically think of for intent taxonomy categories and instead have labels such as Description, Entity, Human, Numeric, and Location.

He achieved excellent results by training a BERT encoder, getting near 90% accuracy in predicting labels for new/unlabeled search keywords.

The big question for me was, could I leverage the same tech (Uber’s Ludwig BERT encoder) to create an accurate model using the search intent labels I’d get from Burkholder’s code?

It turns out the answer is yes!

How to do it

Here’s how the process works:

1. Gather your list of keywords. If you’re planning on training your own model, I recommend doing so within a specific category/niche. Training on clothing-related keywords and then using that model to label finance related keywords will likely be significantly less accurate than training on clothing related keywords and then using that model to label other unlabeled clothing related keywords. That said, I did try using a model labeled on one category/niche to label another, and the results still seemed quite good to me.

2. Run Burkholder’s script over your list of keywords from Step 1. This will require signing up for SerpAPI.com and buying credits. I recommend getting labels for at least 10,000 search queries with this script to use for training. The more training data, the more accurate your model will likely be.

3. Use the labeled data from the previous step as your training data for the BERT model. Batista’s code to do this is very straightforward, and this article will guide you through the process. I was able to get about ~72% accuracy using about 10,000 labels of training data.

4. Use your model from Step 3 to label unlabeled search data, and then take a look at your results!

The results

I ran through this process using a huge list (13,000 keywords) of clothing/fashion-related search terms from SEMrush as my training data. My resulting model gets just about 80% accuracy.

It seems likely that training the model with more data will continue to improve its accuracy up to a point. If any of you attempt it and improve on 80% accuracy, I would love to hear about it. I think with 20,000+ labeled searches, we could see up to maybe 85-90% accuracy.

This means when you ask this model to predict the intent of unlabeled search queries, 8 times out of 10 it will give you the same label as what would have been returned by Burkholder’s Serp API rules-based classifier. It can also do this for free, in large volumes and incredibly fast.

So something that would have taken a few thousand dollars and days of scraping can now be done for free in just minutes.

In my case I used keywords from a related domain (makeup) instead of clothing keywords, and overall I think it did a pretty good job. Labeling 5,000 search queries took under two minutes with the BERT model. Here’s what my results looked like:

The implications

For SEO tools to be useful, they need to be scalable. Keyword research, content strategy, PPC strategy, and SEO strategy usually rely on being able to do analysis across entire niches/themes/topics/websites.

In many industries, the keyword longtails can extend into the millions. So a faster, more affordable approach to Burkholder’s solution can make a lot of difference.

I forsee AI and machine learning tools being used more and more in our industry, enabling SEOs, paid search specialists, and content marketers to gain superpowers that haven’t been possible before these new AI breakthroughs.

Happy analyzing!

Kristin Tynski is a founder and the SVP of Creative at Fractl, a boutique growth agency based in Delray Beach, FL.

Let’s block ads! (Why?)

Big Data – VentureBeat

Compare two plots by finding the minimum distance among points

September 23, 2019 BI News and Info

I have a question about comparing the points within two plots.
I would like to compare two plots and find the minimum distance among their points, in order to find the nearest/common points (i.e. those ones with minimum -or zero-distance) and plot it (overlapping).
What I did is to extract the coordinates of their respectively points. But I do not know how to compare them and/or the two plots. I used the following line of code, but the result is completely different from what I am looking for.

Outer[EuclideanDistance, seq1, seq2, 1] // Flatten

The result should show the points on the plot equal (almost in common) between the two plots.

Could you please help me?
Many thanks,
Val

Let’s block ads! (Why?)

Recent Questions – Mathematica Stack Exchange

Restaurant Interviewing: Strategies for Finding the Right People

August 10, 2019 NetSuite

Posted by Brady Thomason, NetSuite Solution Manager, Restaurant & Hospitality

To say that eating out has become an important part of American culture is putting it lightly. According to the National Restaurant Association (NRA), Americans spend just over half (51%) of their food dollars in restaurants.

That’s more than double the share spent on eating out in 1955, and it’s not a trend that’s slowing down. The NRA further predicts that restaurants will employ 16.9 million Americans by 2029, up from 15.3 million today.

In other words, restaurants are hiring more people than ever. Add in the industry’s eye-popping annual employee turnover rate, which the U.S. Bureau of Labor Statistics pegged at 73% in 2016, and that equals a lot of interviews.

It also means that hiring the right people — those who can handle stress, represent your restaurant, and, perhaps most importantly, stick around a while — can reduce what is becoming a huge interviewing burden.

In fact, having a toolbox of interviewing best practices can help restaurant managers better manage their time, wade through candidates more efficiently, and, ultimately, run a more successful business. Drive this point home with your hiring team: we’re in a people business, and just happen to serve food.

Efficient Interviewing Strategies

Many restaurant positions are entry-level roles, which creates many challenges with finding the right people. Developing a clear interview strategy is imperative to quickly filter candidates and build your “A-team.”

What’s more, finding convenient times to interview candidates can be a challenge in a stressful, fast-moving restaurant environment. For these reasons, alternate interview approaches can prove effective for working through the growing number of candidates.

Some restaurants might consider using open interviews, in which a restaurant simply announces it will be interviewing during a given window of time, and anyone who wants to apply can just show up. Open interviews enable restaurant managers to interview a lot of candidates in a short time, without the need to coordinate appointments for days on end.

Another option is a group interview, in which a restaurant calls in several candidates and interviews them simultaneously, sometimes even having them perform some kind of collaborative task. This is a great way to see how people work under pressure and in team situations.

With both approaches, it’s best to conduct interviews either when the restaurant is closed or during gaps between rushes.

Most often, however, restaurants interview candidates individually, in which case the setting becomes a more important consideration. No restaurant wants to take up a table during peak hours for interviews, so if it’s necessary to conduct interviews while things are hopping, it’s helpful to identify a quiet place—such as a banquet room, an office or some other private space—where interviews can be conducted in confidence.

Ask Important Questions

Most interviews eventually come down to questions and answers. And whether a restaurant is hiring servers, kitchen staff or even managers, having good interview templates with pre-established questions by role can prove effective in not only simplifying the interviewer’s job, but also in ensuring they’re asking appropriate questions from a legal standpoint (check state and local laws) and avoiding discrimination.

Questions restaurant managers should ask prospective employees:

-Why do you want to work in the food service industry? (If a candidate doesn’t know how to answer this, it’s a red flag.)

-What do you know about our restaurant, and what makes you a good fit? (This is a great way to see if a candidate has taken the time to prepare.)

-What does the word “hospitality” mean to you? (A simple question that’s at the heart of good food service.)

-Can you provide an example of how you handle unhappy guests? (No restaurant gets it all right. How employees handle problematic scenarios could determine whether diners become — or remain — regulars.)

-How would you handle conflict with co-workers? (Restaurants are charged environments where paths cross constantly. There’s a significant possibility of conflict on any given day.)

-What were the best/worst restaurant experiences you’ve had, and why? (Candidates will reveal numerous strengths and weaknesses in answering this.)

-What are (or will be) your favorite/least favorite parts about working in restaurants? (This can tell a restaurant much about whether a specific candidate will fit into its culture.)

-How do you deal with high-pressure situations? (Candidates who can’t deal with pressure probably aren’t the best fit for restaurant work.)

-What does it mean to you to be part of a team? (Running a successful restaurant is a team sport.)

-How do you handle occasions when life makes it difficult to show up to work on time? (Hint: Candidates who don’t have an answer for this are much more likely to have punctuality issues.)

Homing in on the Gems

Once an interviewer’s gut feeling says that a candidate has potential, additional questions can help to validate whether the person is a good hire.

For instance, the interviewer might want to ask potential servers how they would handle a guest with a coupon that’s either expired or has unseen limitations (such as not being able to be used with another offer). For a kitchen position, it might make sense to ask how a candidate might deal with a server who’s bringing a lot of guest issues to the kitchen’s attention during a rush.

It’s not the actual answers to such questions that matter as much as how candidates handle unexpected questions that put pressure on them to consider many factors — a parallel for what it’s like working in a busy restaurant during the rush.

Similarly, interviewers also can find out more about how candidates perform under pressure by asking them to prepare a basic dish or role-play certain steps of the guest service experience.

In the end, the goal is for restaurants to hire the best employees who are most likely to stay the longest. If a restaurant’s interviewing best practices aren’t carefully considered to improve the chances of hiring the right people, it’ll have devastating consequences on the entire business.

Let’s block ads! (Why?)

The NetSuite Blog

Restaurant Interviewing: Strategies for Finding the Right People

August 3, 2019 NetSuite