year_limited: error on combining data frames with different column indices

[fleimgruber]

Reproducer:

def test_pv_germany():
    load_dotenv()
    API_KEY = os.getenv("ENTSOE_API_KEY")
    client = EntsoePandasClient(api_key=API_KEY)

    tz = "Europe/Berlin"

    start = pd.Timestamp("20180101", tz=tz)
    end = pd.Timestamp("20190102", tz=tz)

    country_code = "DE"

    df = client.query_generation(
        country_code, start=start, end=end, process_type="A16", psr_type="B16"
    )
    print(df.head())

The problematic statement is

df = pd.concat(frames, sort=True)

where it fails with the error

AttributeError: 'NoneType' object has no attribute 'is_extension'

because frames has two dataframes where column labels do not match

frames[0].columns
MultiIndex([('Solar',  'Actual Aggregated'),
            ('Solar', 'Actual Consumption')],
           )

frames[1].columns
Index(['Solar'], dtype='object')

Now this is tricky and I guess does not come up too often, but do you see an easy fix or did similar issues come up with other queries?

[jpaduart]

Ran into that issue as well … happens when you query over specific transition dates. One way to solve this, is to force (ie construct if necessary) two levels in the columns multi-index. This requires the second column level (Aggregated/Consumption) to be standardized. I have no clue if this is the case…

…

Op 28 mei 2021 om 14:56 heeft fleimgruber ***@***.***> het volgende geschreven:  Reproducer: def test_pv_germany(): load_dotenv() API_KEY = os.getenv("ENTSOE_API_KEY") client = EntsoePandasClient(api_key=API_KEY) tz = "Europe/Berlin" start = pd.Timestamp("20180101", tz=tz) end = pd.Timestamp("20190102", tz=tz) country_code = "DE" df = client.query_generation( country_code, start=start, end=end, process_type="A16", psr_type="B16" ) print(df.head()) The problematic statement is df = pd.concat(frames, sort=True) where it fails with the error AttributeError: 'NoneType' object has no attribute 'is_extension' because frames has two dataframes where column labels do not match frames[0].columns MultiIndex([('Solar', 'Actual Aggregated'), ('Solar', 'Actual Consumption')], ) frames[1].columns Index(['Solar'], dtype='object') Now this is tricky and I guess does not come up too often, but do you see an easy fix or did similar issues come up with other queries? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.