Numba nopython mode pipeline error: Resolving a TypingError issue

Frontend
1

Hi all, I’m trying to get my first parallel processing function working using Numba, but I’m stuck with a TypingError in nopython mode. I adapted my function to speed up array operations, yet I’m getting an error mentioning a non-precise type in my array.

Below is a sample of my adapted code:

import numpy as np
from numba import njit

sample_events = np.array([''])

@njit(parallel=True, nopython=True)
def process_events(event_data, event_list):
    idx = event_data.where(event_data == 'Repair')
    collected = np.array('')
    remaining = event_data[idx:]
    if ('LOMT' in remaining) and ('DIMT' in remaining):
        for ev in remaining:
            if 'FU' not in ev:
                collected.append(ev)
            else:
                break
    if len(collected) >= 5:
        event_list.append(collected)

for item in progress_bar(package_list):
    event_data = np.array(data['Event'].loc[data['Package'] == item].values)
    process_events(event_data, sample_events)

I get an error referring to array(pyobject, 1d, C) during the Numba compilation stage. I’m unsure how to resolve this type issue. Has anyone encountered this problem or know how to fix it? Thanks for your help!

2

hey there! i’ve seen similar problms. numba doesnt like object arrays in nopython mode. try using specific dtypes (np.string_) and preallocate arrays instead of appending during loop. hope it helps!

3

hey climbing monkey! have u considered using numba’s typed list instead? it might solve ur issue. also, curious about ur data structure - why not use pandas dataframe operations instead of numpy arrays? could be more efficient for ur use case. what’s the end goal of ur processing?

4

I’ve encountered similar issues with Numba’s nopython mode. The problem likely stems from using Python objects in your arrays, which Numba can’t handle efficiently. To resolve this, consider using fixed-length string types like np.dtype(‘S20’) for your arrays. Additionally, pre-allocating your arrays with a fixed size instead of dynamically appending can improve performance and compatibility with Numba. It might also be worth exploring Numba’s jitclass for more complex data structures if needed. Remember to profile your code before and after optimizations to ensure you’re actually gaining performance benefits from these changes.