Energy data not preserved when moving from StealthChesnut integration #224

nigelspowage · 2023-05-08T11:29:33Z

The documentation says that when moving from stealthchesnut to this integration "Changes to the energy dashboard will not effect your older energy statistics." however when I have done this my previous energy data is not shown.

I posted an issue on the wiki but I've not yet had any helpful assistance. I realise this might not be an issue with this integration but something related to my HA setup, I would appreciate help in getting to the bottom of this and hopefully getting the 6 months of data displayed again.

FozzieUK · 2023-05-09T19:51:58Z

I’ve never known the energy dashboard statistics to be reset, it’s something a lot of people have asked for and very difficult to manage, but looking at the graph you posted something appears to have gone wrong. Is there any historical data showing in the energy dashboard, pv, battery charge/discharge, grid consumption, export etc..?
Also worth asking the question do you have recent backups, preferably just prior to moving across to this integration in case we need to roll it back and try again without deleting the utility meters, and the old Riemann ‘sum’ entities, then not using the integration to auto-update the dashboard (it should still work on the old settings if the utility meters etc.. are in place)

FozzieUK · 2023-05-09T20:06:15Z

It might also be worth checking the statistics, go to Developer Tools, Statistics and have a look for any entities listed that have the words ‘FIX ISSUE’ next to them - we are particularly interested in the original entity names you used in the dashboard (if you can remember) possibly named pv1_daily, pv2_daily, feedin daily. If they are listed with a fix issue next to them, click it and screenshot what it says on here it’’’ be things like ‘the unit of the entity changed’ or similar.

nigelspowage · 2023-05-10T11:18:58Z

Hi Fozzie, thanks for getting back to me. In answer to your questions:
Is there any historical energy dashboard data - no, all the energy dashboard data stops displaying around 6pm on 2nd May which is when I switched over to this integration.
Backups? - I have been using the google drive backup add-on and I hadn't checked how it was configured. I do have a backup but it's only from after the 2nd (3rd) so I don't have a full backup from prior to switching.
Entities with 'FIX ISSUE' - yes, although I think a number of these are not related to this issue. I originally tried using a plugin for foxess that scraped the foxess portal over a year ago. It didn't work very well and I gave up on it after a few days and removed it. This left behind a number of orphaned entities I never bothered removing. I still have these and they're listed with 'there is no state available for this entity'. I'll ignore those.

pv1_daily - this does have the unit changed error, I presume I ask it to fix the issue by updating the units of the old data?

FozzieUK · 2023-05-10T15:04:21Z

Ok, thanks - I hope not to have to go near the backups but always the first question to ask.

On the pv1_daily there is definitely been a error with the historical unit, earlier HA versions weren't that fussy but for the last 12 months or so it wouldn't have allowed you to configure statistics without fixing it first.

I think before we fix it, as you've added back in the utility meters i'd like as a test for you to go to your energy dashboard (Settings, Dashboards, Energy) and add back in the old entity names - but leave the new ones in place.

So for grid_consumption add grid_daily,
for Return to Grid add feedin_daily
Solar panels would be pv1 & 2_daily
and for the battery Energy going in bat_charge_daily, and Energy coming out would be bat_discharge_daily

Do you still have the original custom sensor sensor.feed_in_power setup in your configuration.yaml - if you do can you change the state line to this -
state: {{ states('sensor.feed_in') | float(default=0) }}

But if you have deleted the old template sensor for feed_in_power, change instead the utility meter for feedin_daily and replace the source with sensor.feed_in

After that, HA will need a restart and then a bit of time to gather it's statistics - i'm hoping that you should start to see some history displayed.

nigelspowage · 2023-05-10T15:54:12Z

Hi,

When trying to add the utility meters back in I get the error 'Pick sensors which measure energy going in to and out of the battery in either of GJ, kWh, MJ, MWh, Wh.'

As suggested for Grid Consumption I'm trying to add 'sensor.grid_daily'.

In my configuration.yaml there's a utility meter configured as

  grid_daily:
    source: sensor.grid_consumption_sum
    cycle: daily

and grid_consumption_sum is defined as a sensor as follows:

  - method: left
    name: grid_consumption_sum
    platform: integration
    round: 2
    source: sensor.grid_consumption
    unit_time: h

and there's a template for grid consumption as follows

      - name: "Grid Consumption"
        device_class: "power"
        unit_of_measurement: "kW"
        state: >
            {% if (states('sensor.grid_ct') | float(default=0) ) < 0 %}
            {{ states('sensor.grid_ct') | float(default=0) * -1 }}
            {% else %}
            0 
            {% endif %}

FozzieUK · 2023-05-10T16:17:55Z

Ok, lets take these one at a time - the grid consumption template shouldn't be there anymore - that will clash with the new integration, can you either remove or comment out that code (leave the utility meter and the_sum - just remove the template sensor as that's provided in the integration), once removed it's best to restart again.

Once it has restarted go to the integrations page for FoxESS modbus and list all the sensors - look for Grid Consumption in the list, click on it and then on the cog in the top right corner, about half way down the page will be the entity_id which should be sensor.grid_consumption but as you had the template in your config it's likely to be sensor.grid_consumption_2 (or similar) - if it is anything other than sensor.grid_consumption change it to that and click UPDATE.

That will correct the entity name and the both the _sum and utility meter should start to record again.

Moving next to Feedin do you have the utility meter (feedin_daily), the Riemann sum (feedin_sum) and template sensor (feed_in_power) in your config, can you post what you have for these?

nigelspowage · 2023-05-10T16:30:06Z

My apologies but I was incorrect in my previous comment (I was looking at the wrong copy of configuration.yaml).

I had previously commented out the grid consumption template as instructed in the migration instructions. The templates section of my configuration.yaml files looks like this

template:
#  - sensor:
#      - name: "Battery Discharge"
#        device_class: "power"
#        unit_of_measurement: "kW"
#        state: >
#            {% if (states('sensor.battery_discharge_power') | float(default=0) ) > 0 %}
#            {{ states('sensor.battery_discharge_power') | float(default=0) * 1 }}
#            {% else %}
#            0 
#            {% endif %}
#      - name: "Battery Charge"
#        device_class: "power"
#        unit_of_measurement: "kW"
#        state: >
#            {% if (states('sensor.battery_discharge_power') | float(default=0) ) < 0 %}
#            {{ states('sensor.battery_discharge_power') | float(default=0) * -1 }}
#            {% else %}
#            0 
#            {% endif %}
      - name: "Feed In Power"
        device_class: "power"
        unit_of_measurement: "kW"
        state: >
            {% if (states('sensor.grid_ct') | float(default=0) ) > 0 %}
            {{ states('sensor.grid_ct') | float(default=0) * 1 }}
            {% else %}
            0 
            {% endif %}
#      - name: "Grid Consumption"
#        device_class: "power"
#        unit_of_measurement: "kW"
#        state: >
#            {% if (states('sensor.grid_ct') | float(default=0) ) < 0 %}
#            {{ states('sensor.grid_ct') | float(default=0) * -1 }}
#            {% else %}
#            0 
#            {% endif %}
      - name: "PV Power Now"
        device_class: "power"
        unit_of_measurement: "kW"
        state: >
              {{ (states('sensor.pv1_power') | float(default=0) + states('sensor.pv2_power') | float(default=0) ) * 1 }}
      - name: "PV Energy Daily"
        device_class: "energy"
        unit_of_measurement: "kWh"
        state: >
              {{ ((states('sensor.pv1_daily') | float(default=0) + states('sensor.pv2_daily') | float(default=0) ) * 1 ) | round(2) }} 
      - name: "System Losses"
        device_class: "power"
        unit_of_measurement: "kW"
        state: >
               {{ ((states('sensor.pv1_power') | float(default=0)
               + states('sensor.pv2_power') | float(default=0)
               + states('sensor.grid_consumption') | float(default=0)
               + states('sensor.battery_discharge') | float(default=0)
               - states('sensor.battery_charge') | float(default=0)
               - states('sensor.feed_in') | float(default=0)
               - states('sensor.load_power') | float(default=0) )) | round(2) }}

I can confirm it is listed as sensor.grid_consumption in the entity settings.

My utility meter section looks like this.

# Utility Meters - Provides meters to be used with the energy dashboard and reset daily
utility_meter:
  load_daily:
    source: sensor.load_sum
    cycle: daily
  pv1_daily:
    source: sensor.pv1_sum
    cycle: daily
  pv2_daily:
    source: sensor.pv2_sum
    cycle: daily
  bat_charge_daily:
    source: sensor.bat_charge_sum
    cycle: daily
  bat_discharge_daily:
    source: sensor.bat_discharge_sum
    cycle: daily
  feedin_daily:
    source: sensor.feedin_sum
    cycle: daily
  grid_daily:
    source: sensor.grid_consumption_sum
    cycle: daily
  losses_daily:
    source: sensor.losses_sum
    cycle: daily

FozzieUK · 2023-05-10T16:39:57Z

ok, thanks - makes sense.

In the config can you change the state of the Feed In Power template to be as follows -

state: {{ states('sensor.feed_in') | float(default=0) }}

Save your config, click Developer Tools, and in the YAML tab click on 'TEMPLATES' to reload that section

This will now use the new integration to feed the sum and utility meter.

Now, just to go back to the Statistics 'Fix Issue' for pv1_daily, can you click on that and choose the update the historical values to kWh without converting

Can I check were you able to add any of the older xxx_daily meters back in to any section of the Energy Dashboard without error ?

nigelspowage · 2023-05-10T17:13:00Z

I've clicked the fix issue and updated the historical data without converting but that doesn't appear to have made any difference.

For grid_daily, feedin_daily and pv1_daily all give the error that the sensor must use GJ, kWh, MJ, MWh, Wh.

FozzieUK · 2023-05-10T17:16:43Z

Go to Developer Tools, States and search for grid_daily, click on it and at the top of the screen it will list all the attributes of the entity

Can you screen shot that part and post here (also for feedin_daily) so I can see what it thinks it is.

nigelspowage · 2023-05-10T17:20:52Z

FozzieUK · 2023-05-10T17:27:11Z

hmmm, both are set correctly? yet they are being rejected by the Energy Dashboard.

I assume the entity listed in energy dashboard for grid consumption is Grid Consumption Today?, can you delete that entity and try and re-add grid_daily

EDIT: Should have added can you choose not to track costs when you try and add it.

nigelspowage · 2023-05-10T17:43:59Z

The entity listed currently in the grid consumption dashboard is Grid Consumption Total (sensor.grid_consumption_energy_total). That's the one from this integration that seems to be working fine.

That has a bar graph icon next to it, where as sensor.grid_daily has a counter style icon. Should I try adding sensor.grid_consumption to the energy dashboard ? That's got the same bar graph icon as the current entry and the entities tab says it's provided by a sensor and not FoxESS - Modbus.

FozzieUK · 2023-05-10T17:45:09Z

Try adding the sum instead so grid_consumption_sum - that's got to be state_class total_increasing

FozzieUK · 2023-05-10T17:46:34Z

the bar graph means total increasing, whereas the counter is a utility meter - both should be acceptable to the dashboard 🤷

nigelspowage · 2023-05-10T17:46:56Z

Hmm, I'm still getting the error that implies the units are wrong.

FozzieUK · 2023-05-10T17:53:17Z

I think the units wrong error maybe a default error message when something unexpected isn't caught (hence the unknown error above).

I'm beginning to worry that there maybe something wrong with the underlying data structures - are you up to date with the latest version of HA?

nigelspowage · 2023-05-10T17:54:48Z

Yes, my install of HA is completely up to date.

FozzieUK · 2023-05-10T18:02:57Z

Can we leave it overnight to let the statistics keep building and let the utility meters reset at midnight, then try again tomorrow to load grid_daily.

I've never come across this before, i'll do a bit of research and see what I can find - if we can't get it to update the dashboard, it may need that database backup to be restored and then start over again with modifying the config and re-load the integration - but this time don't update the dashboard, use the existing utility meters as they are configured.

nigelspowage · 2023-05-10T18:05:32Z

Yes, leaving it overnight is no problem.

Unfortunately I don't believe I've got a database backup from before I switched the integration so if that's needed then I'm out of luck. While it sucks to lose that data it is a lesson in having good backups that I'll have learnt 😃

FozzieUK · 2023-05-10T18:10:59Z

Ouch, ok keep fingers crossed - i’ll see what I can find in the meantime 👍

FozzieUK · 2023-05-11T09:47:05Z

When you get the chance to try again, if it throws the same error - can you screenshot any errors you get at that time from your log

nigelspowage · 2023-05-11T10:10:56Z

I'm still getting the same error, and checking the logs it seems this is appearing at the relevant time.

Logger: homeassistant.components.websocket_api.http.connection
Source: components/energy/data.py:155
Integration: Home Assistant WebSocket API (documentation, issues)
First occurred: 10 May 2023 at 18:09:56 (6 occurrences)
Last logged: 11:06:59

[2591611056] Error handling message: Unknown error (unknown_error) nigel from 192.168.2.233 (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36)
[2591751592] Error handling message: Unknown error (unknown_error) nigel from 192.168.2.233 (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36)
[2579096144] Error handling message: Unknown error (unknown_error) nigel from 192.168.2.233 (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36)
[2587615272] Error handling message: Unknown error (unknown_error) nigel from 192.168.2.233 (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36)
[2635607128] Error handling message: Unknown error (unknown_error) nigel from 192.168.2.233 (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36)
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/connection.py", line 180, in async_handle
    handler(self.hass, self, schema(msg))
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 272, in __call__
    return self._compiled([], data)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 595, in validate_dict
    return base_validate(path, iteritems(data), out)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 387, in validate_mapping
    cval = cvalue(key_path, value)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 229, in _run
    return self._exec(self._compiled, value, path)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 353, in _exec
    v = func(path, v)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 818, in validate_callable
    return schema(data)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 272, in __call__
    return self._compiled([], data)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 636, in validate_sequence
    cval = validate(index_path, value)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 818, in validate_callable
    return schema(data)
  File "/usr/src/homeassistant/homeassistant/helpers/config_validation.py", line 947, in key_value_validator
    return cast(dict[Hashable, Any], value_schemas[key_value](value))
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 272, in __call__
    return self._compiled([], data)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 595, in validate_dict
    return base_validate(path, iteritems(data), out)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 387, in validate_mapping
    cval = cvalue(key_path, value)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 229, in _run
    return self._exec(self._compiled, value, path)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 353, in _exec
    v = func(path, v)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 636, in validate_sequence
    cval = validate(index_path, value)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 229, in _run
    return self._exec(self._compiled, value, path)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 353, in _exec
    v = func(path, v)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 818, in validate_callable
    return schema(data)
  File "/usr/src/homeassistant/homeassistant/components/energy/data.py", line 155, in _flow_from_ensure_single_price
    val["entity_energy_price"] is not None
KeyError: 'entity_energy_price'

FozzieUK · 2023-05-11T10:22:38Z

Can you remember on your old grid consumption whether you had the costs set to None, a fixed price or the value of an entity ?

nigelspowage · 2023-05-11T10:31:36Z

The old grid consumption had the costs set to a fixed price.

FozzieUK · 2023-05-11T10:32:24Z

could you try adding it again but set a fixed unit price

nigelspowage · 2023-05-11T10:36:40Z

Sure, it seems like the same error occurred.

Logger: homeassistant.components.websocket_api.http.connection
Source: components/energy/data.py:155
Integration: Home Assistant WebSocket API (documentation, issues)
First occurred: 10 May 2023 at 18:09:56 (7 occurrences)
Last logged: 11:34:59

[2591751592] Error handling message: Unknown error (unknown_error) nigel from 192.168.2.233 (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36)
[2579096144] Error handling message: Unknown error (unknown_error) nigel from 192.168.2.233 (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36)
[2587615272] Error handling message: Unknown error (unknown_error) nigel from 192.168.2.233 (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36)
[2635607128] Error handling message: Unknown error (unknown_error) nigel from 192.168.2.233 (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36)
[2566378424] Error handling message: Unknown error (unknown_error) nigel from 192.168.2.233 (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36)
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/connection.py", line 180, in async_handle
    handler(self.hass, self, schema(msg))
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 272, in __call__
    return self._compiled([], data)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 595, in validate_dict
    return base_validate(path, iteritems(data), out)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 387, in validate_mapping
    cval = cvalue(key_path, value)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 229, in _run
    return self._exec(self._compiled, value, path)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 353, in _exec
    v = func(path, v)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 818, in validate_callable
    return schema(data)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 272, in __call__
    return self._compiled([], data)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 636, in validate_sequence
    cval = validate(index_path, value)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 818, in validate_callable
    return schema(data)
  File "/usr/src/homeassistant/homeassistant/helpers/config_validation.py", line 947, in key_value_validator
    return cast(dict[Hashable, Any], value_schemas[key_value](value))
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 272, in __call__
    return self._compiled([], data)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 595, in validate_dict
    return base_validate(path, iteritems(data), out)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 387, in validate_mapping
    cval = cvalue(key_path, value)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 229, in _run
    return self._exec(self._compiled, value, path)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 353, in _exec
    v = func(path, v)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 636, in validate_sequence
    cval = validate(index_path, value)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 229, in _run
    return self._exec(self._compiled, value, path)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/validators.py", line 353, in _exec
    v = func(path, v)
  File "/usr/local/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 818, in validate_callable
    return schema(data)
  File "/usr/src/homeassistant/homeassistant/components/energy/data.py", line 155, in _flow_from_ensure_single_price
    val["entity_energy_price"] is not None
KeyError: 'entity_energy_price'

FozzieUK · 2023-05-11T10:43:28Z

can you check Developer Tools, States and do a search for 'cost' - and screenshot any entities that show there (particularly grid_consumption)

nigelspowage · 2023-05-11T10:57:06Z

This is the only cost that appears.

FozzieUK · 2023-05-11T11:07:12Z

It looks like your old energy grid consumption was probably using the sensor grid_consumption_sum rather than the utility meter

It doesn't help much as the error suggests some kind of corruption in the energy configuration, do you have a file browser installed? - the energy config is stored in \config\ .storage\energy - We could do with seeing what is inside that, it'll look something like this -

nigelspowage · 2023-05-11T11:10:38Z

This is the contents of that file.

{
  "version": 1,
  "minor_version": 1,
  "key": "energy",
  "data": {
    "energy_sources": [
      {
        "type": "solar",
        "stat_energy_from": "sensor.pv1_energy_total"
      },
      {
        "type": "solar",
        "stat_energy_from": "sensor.pv2_energy_total"
      },
      {
        "type": "battery",
        "stat_energy_to": "sensor.battery_charge_total",
        "stat_energy_from": "sensor.battery_discharge_total"
      },
      {
        "type": "grid",
        "flow_from": [
          {
            "stat_energy_from": "sensor.grid_consumption_energy_total"
          }
        ],
        "flow_to": [
          {
            "stat_energy_to": "sensor.feed_in_energy_total"
          }
        ],
        "cost_adjustment_day": 0
      }
    ],
    "device_consumption": []
  }
}```

FozzieUK · 2023-05-11T11:23:52Z

My inclination is to say lets delete that file and restart HA.

HA will re-create it when it starts, and then you can try adding the energy monitoring back in - starting with grid_daily

FozzieUK · 2023-05-11T12:51:05Z

Just thinking, if it won't allow deletion (locked) you may need to rename it to energy.bak and then restart.

nigelspowage · 2023-05-11T15:01:56Z

I've done this and it's been (partially) successful 😄 ! Removing the file and rebooting removed the energy sources from the dashboard (as expected) and I re-added both the historical energy sources and the ones provided by this integration. No errors were presented this time 👍

The energy usage data has now appeared for both prior to changing to this integration and since the integration. The solar production since changing to this integration has also appeared. I'm just missing the solar production for the previous integration.

I'll just double check what I added for the previous integrations solar input. It's possible I added the wrong thing.

nigelspowage · 2023-05-11T15:06:05Z

Yep, I'd added sensor.pv1_daily which didn't work, but sensor.pv_total_sum has been added and it now shows the historical solar data 🥇

@FozzieUK thank you very much for your patience and assistance!

FozzieUK · 2023-05-11T15:19:11Z

@nigelspowage Great, glad it's sorted 👍

Would you mind just doing one last thing, if you could screenshot what your \config\ .storage\energy file looks like now - i'm still intrigued how it broke and whether there is something the integration did that caused it, or could have protected against it.

canton7 · 2023-05-11T15:22:34Z

I haven't been watching this closely, but looks like #172 maybe?

It stops you from editing the energy dashboard, but shouldn't (I think?) affect anything else.

Fixed in the beta (and upgrading to the beta will auto-fix it).

FozzieUK · 2023-05-11T15:46:56Z

@canton7 yeah the editing problem looks a lot like #172 - The energy config itself wasn't corrupt, the error message in the logs was for a missing key (entity_energy_price) and looking at the differences between a working file worked out the entity_energy_price must be there and set to Null if not used, glad it's sorted on latest beta.

I might do a quick change for the migration instructions to recommend keeping the old statistic sensors rather than dashboard auto-update - there are too many bugs in the energy dashboard changing over from the original entities (energy units mis-matching etc..)

nigelspowage · 2023-05-11T15:50:40Z

@FozzieUK My energy file looks like this now:

{
  "version": 1,
  "minor_version": 1,
  "key": "energy",
  "data": {
    "energy_sources": [
      {
        "type": "grid",
        "flow_from": [
          {
            "stat_energy_from": "sensor.grid_daily",
            "stat_cost": null,
            "entity_energy_price": null,
            "number_energy_price": null
          }
        ],
        "flow_to": [
          {
            "stat_energy_to": "sensor.feedin_daily",
            "stat_compensation": null,
            "entity_energy_price": null,
            "number_energy_price": null
          }
        ],
        "cost_adjustment_day": 0.0
      },
      {
        "type": "battery",
        "stat_energy_from": "sensor.battery_discharge_total",
        "stat_energy_to": "sensor.battery_charge_total"
      },
      {
        "type": "battery",
        "stat_energy_from": "sensor.bat_discharge_daily",
        "stat_energy_to": "sensor.bat_charge_daily"
      },
      {
        "type": "solar",
        "stat_energy_from": "sensor.pv_total_sum",
        "config_entry_solar_forecast": null
      }
    ],
    "device_consumption": []
  }
}

nigelspowage · 2023-05-11T15:54:53Z

A few minutes ago I had added both this integrations sensors and the previous ones to the energy dashboard but then quickly noticed I was getting double entries for the new data. I have removed the sensors for this integration from the energy dashboard and the graphs look good now.

canton7 · 2023-05-11T15:55:53Z

Sweet. I was going to ask whether there were any actions we needed to take from this to change the docs. Please makd any improvements you see fit!

FozzieUK · 2023-05-11T16:39:18Z

@nigelspowage - well spotted, as we made changes to get the original sensors working again, it doesn't need the integration ones - they are duplicated going forwards (but the old ones hold history), however statistics don't take up much storage space. :)

@canton7 - i've updated the migration instructions to make the process smoother ;)

nigelspowage closed this as completed May 11, 2023

Energy data not preserved when moving from StealthChesnut integration #224

Energy data not preserved when moving from StealthChesnut integration #224

Comments

nigelspowage commented May 8, 2023 • edited Loading

FozzieUK commented May 9, 2023

FozzieUK commented May 9, 2023

nigelspowage commented May 10, 2023

FozzieUK commented May 10, 2023

nigelspowage commented May 10, 2023

FozzieUK commented May 10, 2023 • edited Loading

nigelspowage commented May 10, 2023

FozzieUK commented May 10, 2023

nigelspowage commented May 10, 2023

FozzieUK commented May 10, 2023

nigelspowage commented May 10, 2023

FozzieUK commented May 10, 2023 • edited Loading

nigelspowage commented May 10, 2023

FozzieUK commented May 10, 2023

FozzieUK commented May 10, 2023

nigelspowage commented May 10, 2023

FozzieUK commented May 10, 2023

nigelspowage commented May 10, 2023

FozzieUK commented May 10, 2023

nigelspowage commented May 10, 2023

FozzieUK commented May 10, 2023

FozzieUK commented May 11, 2023

nigelspowage commented May 11, 2023

FozzieUK commented May 11, 2023

nigelspowage commented May 11, 2023

FozzieUK commented May 11, 2023

nigelspowage commented May 11, 2023

FozzieUK commented May 11, 2023

nigelspowage commented May 11, 2023

FozzieUK commented May 11, 2023 • edited Loading

nigelspowage commented May 11, 2023

FozzieUK commented May 11, 2023

FozzieUK commented May 11, 2023

nigelspowage commented May 11, 2023

nigelspowage commented May 11, 2023

FozzieUK commented May 11, 2023

canton7 commented May 11, 2023 • edited Loading

FozzieUK commented May 11, 2023

nigelspowage commented May 11, 2023 • edited Loading

nigelspowage commented May 11, 2023

canton7 commented May 11, 2023

FozzieUK commented May 11, 2023

nigelspowage commented May 8, 2023 •

edited

Loading

FozzieUK commented May 10, 2023 •

edited

Loading

FozzieUK commented May 10, 2023 •

edited

Loading

FozzieUK commented May 11, 2023 •

edited

Loading

canton7 commented May 11, 2023 •

edited

Loading

nigelspowage commented May 11, 2023 •

edited

Loading