Concatenate generator and value

小能豆

Concatenate generator and value

I have to pass a generator with a value to specific function as an argument.

function Func inputs pd.DataFrame and a value.

def Func(df:pd.DataFrame, option:int):
    abcd

and I want to run this function to all sub-dataframes.

from pqdm.processes import pqdm

results = pqdm((DataFrame.groupby(["col1", "col2"]), 3) , Func, n_jobs=16)
                #^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                 # this part is the problem. It have to a single generator.

“DataFrame.groupby([“col1”, “col2”])” is generator. and 3 is just a value.

How can I concat a generator and a value?

I need a new generator which returns (subgroup1,3), (subgroup2,3),(subgroup3,3),(subgroup4,3),.....

阅读 74

2023-12-12

共1个答案

小能豆

To achieve this, you can create a custom generator that pairs each subgroup from the grouped DataFrame with the specified value. Here’s an example of how you can do this:

from pqdm.processes import pqdm
import pandas as pd

def Func(df: pd.DataFrame, option: int):
    # Your implementation of Func
    # ...

# Sample DataFrame
data = {'col1': ['A', 'A', 'B', 'B', 'C', 'C'],
        'col2': [1, 2, 1, 2, 1, 2],
        'value': [10, 20, 30, 40, 50, 60]}

df = pd.DataFrame(data)

# Custom generator function
def grouped_generator(df_grouped, value):
    for name, group in df_grouped:
        yield group, value

# Create the generator
grouped_gen = grouped_generator(df.groupby(["col1", "col2"]), 3)

# Use pqdm with the custom generator
results = pqdm(grouped_gen, Func, n_jobs=16)

In this example, the grouped_generator function takes a grouped DataFrame and a value as input and yields pairs of (subgroup, value) for each subgroup. Then, you can use pqdm with this generator.

Adjust the Func implementation according to your requirements. The key here is to create a custom generator that pairs each subgroup with the specified value.

2023-12-12