How can I determine which Parquet engine Pandas is using?

Frontend and Backend Development
1

I know that Pandas supports working with Parquet files through various engine options—in particular, the libraries pyarrow and fastparquet. I’m running an Intel Conda distribution where calls to the DataFrame export method work flawlessly, even though I haven’t installed pyarrow. It appears that an alternative engine is handling the task, but how can I confirm which one is active?

Below is an example snippet that illustrates a potential approach:

import pandas as pd

def check_engine():
    sample_df = pd.DataFrame({'colA': [10, 20], 'colB': [30, 40]})
    # The following line simulates an export using a custom method to indirectly determine the engine
    engine_used = sample_df.save_as_parq('sample_output.parq', engine='alt_backend')
    return engine_used

print(check_engine())
2

A reliable way to confirm which Parquet engine is being used is to verify which library modules are available in the environment. From personal experience, attempting to import pyarrow and fastparquet separately helps determine which one is installed and being leveraged during file operations. Additionally, setting the engine explicitly in your export functions can pinpoint the source in the resulting behavior. It is also practical to enable detailed logging or run in a debug mode to observe internal decisions made by Pandas when selecting the applicable engine.