Query to generation per plant does not work

[huertamir]

Other queries work but when calling to the query to generation per plant the system hangs.

[fboerman]

hi @huertamir could you please provide a code sample?

[ASamarkRoth]

Not sure I experience the same as @huertamir, but I also have problems reading generation per plant with the pandas client:

from entsoe import EntsoeRawClient, EntsoePandasClient, Area
import pandas as pd
client = EntsoePandasClient(api_key=api_key)
start = pd.to_datetime("31.10.2020")
end = start+pd.Timedelta("24H")
client.query_generation_per_plant(country_code=Area.SE_1, start=start, end=end)

I get the following error:

---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
~/miniconda3/lib/python3.8/site-packages/entsoe/entsoe.py in _base_request(self, params, start, end)
    119         try:
--> 120             response.raise_for_status()
    121         except requests.HTTPError as e:

~/miniconda3/lib/python3.8/site-packages/requests/models.py in raise_for_status(self)
    942         if http_error_msg:
--> 943             raise HTTPError(http_error_msg, response=self)
    944 

HTTPError: 400 Client Error: Bad Request for url: https://transparency.entsoe.eu/api?documentType=A73&processType=A16&in_Domain=10Y1001A1001A44P&securityToken={api_key}&periodStart=202010310000&periodEnd=202011010000

During handling of the above exception, another exception occurred:

InvalidBusinessParameterError             Traceback (most recent call last)
/tmp/ipykernel_30797/1680030421.py in <module>
      4 print(start, end)
      5 
----> 6 client.query_generation_per_plant(country_code=Area.SE_1, start=start, end=end)

~/miniconda3/lib/python3.8/site-packages/entsoe/entsoe.py in day_wrapper(start, end, *args, **kwargs)
    987         for _start, _end in blocks:
    988             try:
--> 989                 frame = func(*args, start=_start, end=_end, **kwargs)
    990             except NoMatchingDataError:
    991                 print(f"NoMatchingDataError: between {_start} and {_end}")

~/miniconda3/lib/python3.8/site-packages/entsoe/entsoe.py in query_generation_per_plant(self, country_code, start, end, psr_type, include_eic, nett, **kwargs)
   1767         """
   1768         area = lookup_area(country_code)
-> 1769         text = super(EntsoePandasClient, self).query_generation_per_plant(
   1770             country_code=area, start=start, end=end, psr_type=psr_type)
   1771         df = parse_generation(text, per_plant=True, include_eic=include_eic)

~/miniconda3/lib/python3.8/site-packages/entsoe/entsoe.py in query_generation_per_plant(self, country_code, start, end, psr_type, **kwargs)
    368         if psr_type:
    369             params.update({'psrType': psr_type})
--> 370         response = self._base_request(params=params, start=start, end=end)
    371         return response.text
    372 

~/miniconda3/lib/python3.8/site-packages/entsoe/entsoe.py in retry_wrapper(*args, **kwargs)
     37         for _ in range(self.retry_count):
     38             try:
---> 39                 result = func(*args, **kwargs)
     40             except (requests.ConnectionError, gaierror) as e:
     41                 error = e

~/miniconda3/lib/python3.8/site-packages/entsoe/entsoe.py in _base_request(self, params, start, end)
    127                     raise NoMatchingDataError
    128                 elif "check you request against dependency tables" in error_text:
--> 129                     raise InvalidBusinessParameterError
    130                 elif "is not valid for this area" in error_text:
    131                     raise InvalidPSRTypeError

InvalidBusinessParameterError: 

The data is there on the web client:

When requesting Area.SE, Area.NO or Area.FI instead it works. It does not work with Area.DK, and neither Area.NO_1 etc.

I am running version 0.4:

import entsoe
entsoe.__version__
```bash
'0.4'

[aliciabassiere]

I have the same issue, also for DK.

I think the problem comes from the fact that the query seems to use control_area instead of a bidding zone.

HTTPError Traceback (most recent call last)

~\miniconda3\lib\site-packages\entsoe\entsoe.py in _base_request(self, params, start, end)
92 try:
---> 93 response.raise_for_status()
94 except requests.HTTPError as e:

~\miniconda3\lib\site-packages\requests\models.py in raise_for_status(self)
942 if http_error_msg:
--> 943 raise HTTPError(http_error_msg, response=self)
944

HTTPError: 400 Client Error: Bad Request for url: https://transparency.entsoe.eu/api?documentType=A73&processType=A16&in_Domain=10Y1001A1001A83F&securityToken="TOKEN"&periodStart=201412312300&periodEnd=201501012300

[fedeguerr]

Sorry for writing on a quite old post, but I encountered the same issue in version 0.5.4 with Italian BZN's.
I can access data requesting IT, but I obtain the error mentioned above when requesting IT_NORD or any of the other Italian BZN's. I also tried to use the Area.BZN form, and also their corresponding codes, still no success.

It is quite surprising, as this error seems to occur only when calling generation_per_plant: I had no issues requesting IT_NORD to query_aggregate_water_reservoirs_and_hydro_storage or query_generation, for instance.

[fboerman]

hi @fedeguerr fine that you react to old post if it has new info! this is very puzzling indeed. so you are saying that even with control_area you cant get it to work?

[blacksocket]

Hello All,

I am having the same issue below when calling documentType=A73&processType=A16&in_Domain=10YPL-AREA-----S&psrType=B16. The data are indeed visible on entsoe webpage. Also the same query for different psrType works properly like i.e. B02.

2022-10-01 00:00:00+02:00
B16
Traceback (most recent call last):
File "/home/pi/.local/lib/python3.7/site-packages/entsoe/entsoe.py", line 93, in _base_request
response.raise_for_status()
File "/usr/local/lib/python3.7/dist-packages/requests/models.py", line 960, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://transparency.entsoe.eu/api?documentType=A73&processType=A16&in_Domain=10YPL-AREA-----S&psrType=B16&securityToken=d059c8b6-b0c4-4462-8d03-89ec15557100&periodStart=202209302200&periodEnd=202210012200

[echoqsun]

Hello All,

I had an issue when I query data client.query_generation_per_plant('UA_BEI', start=start, end=end, nett=True, include_eic=True),
It raised this kind of error, and it seems from the parse error, But ENTSOE-PT has this data.

How can we figure out this issue?

[fboerman]

hi @blacksocket I am not very well versed in those business parameters. Could you explain to me the difference of what call works and what doesnt? also do not post your api key please.

hi @echoqsun the crash only shows up for include_eic=True. I am trying to get to the bottom of it but havent found the bug yet. will update you when I do

[sereksim]

For me the query does not work either. It is not possible to get the data for Germany (DE_LU or DE_AT_LU)

start = pd.Timestamp('20201201', tz='Europe/Brussels')
end = pd.Timestamp('20201202', tz='Europe/Brussels')
client.query_generation_per_plant('DE_LU', start=start, end=end,psr_type=None, include_eic=False)

throws an InvalidBusinessParameterError:

Traceback

Traceback (most recent call last):
  File "C:\Users\...\miniconda3\lib\site-packages\entsoe\entsoe.py", line 98, in _base_request
    response.raise_for_status()
  File "C:\Users\...\miniconda3\lib\site-packages\requests\models.py", line 960, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://web-api.tp.entsoe.eu/api?documentType=A73&processType=A16&in_Domain=10Y1001A1001A82H&securityToken=...&periodStart=202011302300&periodEnd=202012012300

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\...\miniconda3\lib\site-packages\entsoe\decorators.py", line 125, in day_wrapper
    frame = func(*args, start=_start, end=_end, **kwargs)
  File "C:\Users\...\miniconda3\lib\site-packages\entsoe\entsoe.py", line 1952, in query_generation_per_plant
    text = super(EntsoePandasClient, self).query_generation_per_plant(
  File "C:\Users\...\miniconda3\lib\site-packages\entsoe\entsoe.py", line 367, in query_generation_per_plant
    response = self._base_request(params=params, start=start, end=end)
  File "C:\Users\...\miniconda3\lib\site-packages\entsoe\decorators.py", line 22, in retry_wrapper
    result = func(*args, **kwargs)
  File "C:\Users\...\miniconda3\lib\site-packages\entsoe\entsoe.py", line 107, in _base_request
    raise InvalidBusinessParameterError
entsoe.exceptions.InvalidBusinessParameterError
Breakpoint reached: Any exception
Stack: 
	_base_request, entsoe.py:107
	retry_wrapper, decorators.py:22
	query_generation_per_plant, entsoe.py:367
	query_generation_per_plant, entsoe.py:1952
	day_wrapper, decorators.py:125
	<module>, entsoe_api.py:22
python-BaseException

start = pd.Timestamp('20161215', tz='Europe/Brussels')
end = pd.Timestamp('20161216', tz='Europe/Brussels')
client.query_generation_per_plant('DE_AT_LU', start=start, end=end, psr_type=None, include_eic=False)

throws the same:

Stacktrace

Traceback (most recent call last):
  File "C:\Users\...\miniconda3\lib\site-packages\entsoe\entsoe.py", line 98, in _base_request
    response.raise_for_status()
  File "C:\Users\...\miniconda3\lib\site-packages\requests\models.py", line 960, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://web-api.tp.entsoe.eu/api?documentType=A73&processType=A16&in_Domain=10Y1001A1001A63L&securityToken=...&periodStart=201612142300&periodEnd=201612152300

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\...\miniconda3\lib\site-packages\entsoe\decorators.py", line 125, in day_wrapper
    frame = func(*args, start=_start, end=_end, **kwargs)
  File "C:\Users\...\miniconda3\lib\site-packages\entsoe\entsoe.py", line 1952, in query_generation_per_plant
    text = super(EntsoePandasClient, self).query_generation_per_plant(
  File "C:\Users\...\miniconda3\lib\site-packages\entsoe\entsoe.py", line 367, in query_generation_per_plant
    response = self._base_request(params=params, start=start, end=end)
  File "C:\Users\...\miniconda3\lib\site-packages\entsoe\decorators.py", line 22, in retry_wrapper
    result = func(*args, **kwargs)
  File "C:\Users\...\miniconda3\lib\site-packages\entsoe\entsoe.py", line 107, in _base_request
    raise InvalidBusinessParameterError
entsoe.exceptions.InvalidBusinessParameterError
Breakpoint reached: Any exception
Stack: 
	_base_request, entsoe.py:107
	retry_wrapper, decorators.py:22
	query_generation_per_plant, entsoe.py:367
	query_generation_per_plant, entsoe.py:1952
	day_wrapper, decorators.py:125
	<module>, entsoe_api.py:25
python-BaseException

---

On the other hand, e.g. AT (Austria) works without problems:

start = pd.Timestamp('20201201', tz='Europe/Brussels')
end = pd.Timestamp('20210101', tz='Europe/Brussels')
client.query_generation_per_plant('AT', start=start, end=end, psr_type=None, include_eic=False)

[sereksim]

@fboerman

the crash only shows up for include_eic=True

As you can see in the code i posted above, the error appears with include_eic=False too

[Bert-Herteleer]

From electricitymaps/ENTSOE.py the comment states that generation per unit is only at EIC level:

Generation per unit can only be obtained at EIC (Control Area) level

The list of EICs can be found at: ENTSO-E transparency API or look at ENTSOE_DOMAIN_MAPPINGS in the Electricity Maps file.

See also the [Postman API documentation])https://documenter.getpostman.com/view/7009892/2s93JtP3F6#537e921c-b261-46d1-9515-32958e8f520a), which also indicates that the EIC code must be used. In some cases, this means that the "easy" country code must be replaced by the more involved EIC code. For the example of Belgium:
country_code = 'BE' does not work, whereas EIC_code = '10YBE----------2' used as client.query_generation(EIC_code,start=start, end=end) works.