Searching or checking an element in a large Numpy array using "in" is very slow

Question

I have a very large Numpy array of size 750K. In my code, I am trying to check elements in the array using the following logic:

if (vals[0] in mv_list) #mv_list is numpy array

It has been more than 30 minutes and it could not find ~100,000 elements. How can I speed up the search logic?

Answer 1

Checking an element in a list or Numpy array using "in" clause is extremely slow. You should convert Numpy array to Set to speed up the search logic.

Make the following changes in your code and hopefully, it will fix the issue.

mv_list = set(mv_list)

if (vals[0] in mv_list) #mv_list is numpy array

Also, make sure mv_list = set(mv_list) is not inside any loop, otherwise, it will also become slow.