Q:

Create dataframe from itertools product

I have two lists:
a = [1,2,3]
b = [4,5,6]

I want to create a dataframe whereby each combination of (a,b) generates a dataframe X and I pick out the max value of X, with the resulting output rows/columns with the elements in a and b.
df=[]

for i, j in itertools.product(a, b):
    X = do_something(i,j)  ## this is a dataframe
    x_value = X.max()
    df.append(i,j,x_value)

df=pd.DataFrame(df, columns=['a', 'b', 'x_value'])

The output dataframe should have columns as a, rows as b, and values as x_value.
    1   2   3
4           
5           
6           

A:

IIUC, you want to know how to go from a list of (i, j, x) values to a DataFrame where i corresponds to the columns, j the index, and x the value:
For example, if you had:
a = [1,2,3]
b = [4,5,6]
func = lambda i, j: i+j
result = [(i, j, func(i,j)) for i, j in itertools.product(a, b)]
print(result)
#[(1, 4, 5),
# (1, 5, 6),
# (1, 6, 7),
# (2, 4, 6),
# (2, 5, 7),
# (2, 6, 8),
# (3, 4, 7),
# (3, 5, 8),
# (3, 6, 9)]

One way to turn this into a DataFrame is to use collections.defaultdict:
from collections import defaultdict

d = defaultdict(list)

for i, j, x in result:
    d[i].append(x)

df = pd.DataFrame(d, index=b)
print(df)
#   1  2  3
#4  5  6  7
#5  6  7  8
#6  7  8  9

