Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A couple API Endpoints #10658

Merged
merged 12 commits into from
Aug 19, 2024
Merged

A couple API Endpoints #10658

merged 12 commits into from
Aug 19, 2024

Conversation

Myoldmopar
Copy link
Member

Pull request overview

  • Adds an endpoint to get current IDF file path back out of the API
  • Adds ability to interact with EMS global variables from the API

This won't affect any existing API usage, just adds a few new endpoints.

@Myoldmopar Myoldmopar added this to the EnergyPlus 24.2 IOFreeze milestone Aug 12, 2024
@Myoldmopar Myoldmopar added the NewFeature Includes code to add a new feature to EnergyPlus label Aug 12, 2024
Comment on lines 243 to 249
char *inputFilePath(EnergyPlusState state)
{
const auto *thisState = static_cast<EnergyPlus::EnergyPlusData *>(state);
char *p = new char[std::strlen(thisState->dataStrGlobals->inputFilePath.string().c_str()) + 1];
std::strcpy(p, thisState->dataStrGlobals->inputFilePath.string().c_str());
return p;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, no problem!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment on lines +128 to +129
/// \return A char * of the input file path. This allocates a new char *, and calling clients must free this when done with it!
ENERGYPLUSLIB_API char *inputFilePath(EnergyPlusState state);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

User is warned this allocates, good

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes....but now I think I need to add something to the Python API. Once Python calls this function and converts it into a Python string (PyObject*), the C string is still hanging out. I may need a very minimal function in the C API that can free one of these char*. Thoughts @jmarrec ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can to decode it to utf-8, then just free it, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as

list_response = [r[i].decode('utf-8', errors='ignore') for i in range(count.value)]
self.api.freeObjectNames(r, count) # free the underlying C memory now that we have a Python copy

Copy link
Contributor

@jmarrec jmarrec Aug 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Speaking off, you should:

  • Use FileSystem::toGenericString so you don't have issues with special chars
  • (Would be nice if the python one would return a pathlib.Path instead)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 2c31f9f.

And you don't need to free the c_char_p returned by the C fun: ,if you set restype to c_char_p, ctypes returns a regular Python bytes object. and it seems it manages the memory for you. I tested with and without a C delete[] call, with ASAN enabled, and it works correctly when you do nothing, and it gives a malloc error if you try to free.

> /Users/julien/Software/Others/EnergyPlus-build/Products/pyenergyplus/datatransfer.py(377)get_input_file_path()
-> c_string = self.api.inputFilePath(state)
(Pdb) n
> /Users/julien/Software/Others/EnergyPlus-build/Products/pyenergyplus/datatransfer.py(378)get_input_file_path()
-> res = Path(c_string.decode('utf-8'))
(Pdb) p c_srtring
*** NameError: name 'c_srtring' is not defined
(Pdb) p c_string
b'/Users/julien/Software/Others/EnergyPlus/testfiles/_1ZoneUncontrolled_ForAPITesting.idf'
(Pdb) type(c_string)
<class 'bytes'>
(Pdb) n
> /Users/julien/Software/Others/EnergyPlus-build/Products/pyenergyplus/datatransfer.py(379)get_input_file_path()
-> self.api.freeCString(c_string)
(Pdb) s
=================================================================
==47091==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x0001072b28d0 in thread T0
    #0 0x1016f5c88 in wrap__ZdaPv+0x74 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x61c88)
    #1 0x116cd5a04 in freeCString datatransfer.cc:293
    #2 0x19b79504c in ffi_call_SYSV+0x4c (libffi.dylib:arm64e+0x804c)
    #3 0x19b79dadc in ffi_call_int+0x4b8 (libffi.dylib:arm64e+0x10adc)
    #4 0x1068b3ef4 in _ctypes_callproc+0x530 (_ctypes.cpython-39-darwin.so:arm64+0xbef4)
    #5 0x1068ad238 in PyCFuncPtr_call+0x48c (_ctypes.cpython-39-darwin.so:arm64+0x5238)
    #6 0x1021d3584 in _PyObject_MakeTpCall+0x164 (libpython3.9.dylib:arm64+0x3f584)
    #7 0x1022a9620 in call_function+0x1fc (libpython3.9.dylib:arm64+0x115620)
    #8 0x1022a6c38 in _PyEval_EvalFrameDefault+0x5a24 (libpython3.9.dylib:arm64+0x112c38)
    #9 0x1021d3d7c in function_code_fastcall+0x6c (libpython3.9.dylib:arm64+0x3fd7c)
    #10 0x1022a95c0 in call_function+0x19c (libpython3.9.dylib:arm64+0x1155c0)
    #11 0x1022a6c14 in _PyEval_EvalFrameDefault+0x5a00 (libpython3.9.dylib:arm64+0x112c14)
    #12 0x1021d3d7c in function_code_fastcall+0x6c (libpython3.9.dylib:arm64+0x3fd7c)
    #13 0x1068b31b4 in closure_fcn+0x21c (_ctypes.cpython-39-darwin.so:arm64+0xb1b4)
    #14 0x19b79df2c in ffi_closure_SYSV_inner+0x330 (libffi.dylib:arm64e+0x10f2c)
    #15 0x19b7951e4 in ffi_closure_SYSV+0x34 (libffi.dylib:arm64e+0x81e4)
    #16 0x116c99d68 in decltype(std::declval<void (*&)(void*)>()(std::declval<void*>())) std::__1::__invoke[abi:ue170006]<void (*&)(void*), void*>(void (*&)(void*), void*&&) invoke.h:340
    #17 0x116c99c94 in void std::__1::__invoke_void_return_wrapper<void, true>::__call[abi:ue170006]<void (*&)(void*), void*>(void (*&)(void*), void*&&) invoke.h:415
    #18 0x116c99c68 in std::__1::__function::__alloc_func<void (*)(void*), std::__1::allocator<void (*)(void*)>, void (void*)>::operator()[abi:ue170006](void*&&) function.h:193
    #19 0x116c95a44 in std::__1::__function::__func<void (*)(void*), std::__1::allocator<void (*)(void*)>, void (void*)>::operator()(void*&&) function.h:364
    #20 0x116c77aa4 in std::__1::__function::__value_func<void (void*)>::operator()[abi:ue170006](void*&&) const function.h:518
    #21 0x116c778a8 in std::__1::function<void (void*)>::operator()(void*) const function.h:1169
    #22 0x11af8c4f8 in EnergyPlus::PluginManagement::runAnyRegisteredCallbacks(EnergyPlus::EnergyPlusData&, EnergyPlus::EMSManager::EMSCallFrom, bool&) PluginManager.cc:142
    #23 0x1184ecd78 in EnergyPlus::EMSManager::ManageEMS(EnergyPlus::EnergyPlusData&, EnergyPlus::EMSManager::EMSCallFrom, bool&, ObjexxFCL::Optional<int const>) EMSManager.cc:276
    #24 0x119816ab8 in EnergyPlus::HeatBalanceManager::ManageHeatBalance(EnergyPlus::EnergyPlusData&) HeatBalanceManager.cc:218
    #25 0x11b98a644 in EnergyPlus::SimulationManager::ManageSimulation(EnergyPlus::EnergyPlusData&) SimulationManager.cc:497
    #26 0x116d1b794 in runEnergyPlusAsLibrary(EnergyPlus::EnergyPlusData&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&) EnergyPlusPgm.cc:458
    #27 0x116c7f924 in energyplus runtime.cc:76
    #28 0x19b79504c in ffi_call_SYSV+0x4c (libffi.dylib:arm64e+0x804c)
    #29 0x19b79dadc in ffi_call_int+0x4b8 (libffi.dylib:arm64e+0x10adc)
    #30 0x1068b3ef4 in _ctypes_callproc+0x530 (_ctypes.cpython-39-darwin.so:arm64+0xbef4)
    #31 0x1068ad238 in PyCFuncPtr_call+0x48c (_ctypes.cpython-39-darwin.so:arm64+0x5238)
    #32 0x1021d3584 in _PyObject_MakeTpCall+0x164 (libpython3.9.dylib:arm64+0x3f584)
    #33 0x1022a9620 in call_function+0x1fc (libpython3.9.dylib:arm64+0x115620)
    #34 0x1022a6c38 in _PyEval_EvalFrameDefault+0x5a24 (libpython3.9.dylib:arm64+0x112c38)
    #35 0x1021d3d7c in function_code_fastcall+0x6c (libpython3.9.dylib:arm64+0x3fd7c)
    #36 0x1022a95c0 in call_function+0x19c (libpython3.9.dylib:arm64+0x1155c0)
    #37 0x1022a6c14 in _PyEval_EvalFrameDefault+0x5a00 (libpython3.9.dylib:arm64+0x112c14)
    #38 0x1022aa4e8 in _PyEval_EvalCode+0xba8 (libpython3.9.dylib:arm64+0x1164e8)
    #39 0x1022a1148 in PyEval_EvalCode+0x4c (libpython3.9.dylib:arm64+0x10d148)
    #40 0x1022e8f98 in pyrun_file+0x12c (libpython3.9.dylib:arm64+0x154f98)
    #41 0x1022e6f6c in PyRun_SimpleFileExFlags+0x26c (libpython3.9.dylib:arm64+0x152f6c)
    #42 0x102303f40 in Py_RunMain+0x6a0 (libpython3.9.dylib:arm64+0x16ff40)
    #43 0x102304404 in pymain_main+0x150 (libpython3.9.dylib:arm64+0x170404)
    #44 0x102304480 in Py_BytesMain+0x24 (libpython3.9.dylib:arm64+0x170480)
    #45 0x18a4460dc  (<unknown module>)

Address 0x0001072b28d0 is a wild pointer inside of access range of size 0x000000000001.
SUMMARY: AddressSanitizer: bad-free (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x61c88) in wrap__ZdaPv+0x74
==47091==ABORTING
Abort trap: 6

{
auto *thisState = static_cast<EnergyPlus::EnergyPlusData *>(state);
int index = 0;
for (auto const &erlVar : thisState->dataRuntimeLang->ErlVariable) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you really sure this is the right array, at least unbounded whatsoever like this? More on this in a bit

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great catch, so now I am skipping the built-in EMS variables. If someone wants to get one of those built-in variables, they can get them through normal API methods.

Comment on lines 614 to 627
void setEMSGlobalVariableValue(EnergyPlusState state, int handle, Real64 value)
{
auto *thisState = static_cast<EnergyPlus::EnergyPlusData *>(state);
if (handle < 0 || handle > thisState->dataRuntimeLang->NumErlVariables) {
// need to fatal out once the plugin is done
// throw an error, set the fatal flag, and then return
EnergyPlus::ShowSevereError(
*thisState, fmt::format("Data Exchange API: Problem -- index error in setEMSGlobalVariableValue; received handle: {}", handle));
EnergyPlus::ShowContinueError(*thisState,
"The setEMSGlobalVariableValue function will return to allow the plugin to finish, then EnergyPlus will abort");
thisState->dataPluginManager->apiErrorFlag = true;
}
thisState->dataRuntimeLang->ErlVariable(handle).Value.Number = value;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, I'm pretty sure this is not ok.

ErlVariable array contains a bunch of builtins that you aren't supposed to be able to mess with. NullVariableNum, YEAR, FALSE, TRUE, PI, ACTUALTIME, etc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you haven't defined any EMS stuff in your IDF, this is what the ErlVariable array contains:

1 - NULL
2 - FALSE
3 - TRUE
4 - OFF
5 - ON
6 - PI
7 - TIMESTEPSPERHOUR
8 - YEAR
9 - CALENDARYEAR
10 - MONTH
11 - DAYOFMONTH
12 - DAYOFWEEK
13 - DAYOFYEAR
14 - HOUR
15 - TIMESTEPNUM
16 - MINUTE
17 - HOLIDAY
18 - DAYLIGHTSAVINGS
19 - CURRENTTIME
20 - SUNISUP
21 - ISRAINING
22 - SYSTEMTIMESTEP
23 - ZONETIMESTEP
24 - CURRENTENVIRONMENT
25 - ACTUALDATEANDTIME
26 - ACTUALTIME
27 - WARMUPFLAG

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of these (except the constants such as TRUE, FALSE) are being assigned a value in InitializeRuntimeLanguage

state.dataRuntimeLang->ErlVariable(state.dataRuntimeLangProcessor->YearVariableNum).Value = SetErlValueNumber(double(state.dataEnvrn->Year));

So a change would be therefore limited to any calling points after the point where you are setting a new value. (eg change that in BeginNewEnvironement calling point).

Not sure whether that makes a ton of sense. If not, maybe considering slicing the ErlVariable array to ommit the built in ones, so the user ones only are left.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

first 27 values in ErlVariable() are constant and built-in variables

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EMS Sensors, Actuators and InternalVariables declared in the IDF are actually emplaced before the built-in ones.


GetEMSInput is called before

VariableNum = RuntimeLanguageProcessor::NewEMSVariable(state, cAlphaArgs(1), 0);

So Sensors, Actuators and InternalVariables are loaded in the array first, and only then the InitializeRuntimeLanguage is called

RuntimeLanguageProcessor::InitializeRuntimeLanguage(
state); // Loads built-in globals and functions, then performs GetInput for runtime language objects

Comment on lines 110 to 112
char *filePath = inputFilePath(state);
printf("Input file path accessed via API: %s\n", filePath);
free(filePath);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, but where's the getEMSGlobalVariableValue / setEMSGlobalVariableValue test at?

(and I suppose the python version would be nice too).

Copy link
Member Author

@Myoldmopar Myoldmopar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, this seems very close now. Thanks @jmarrec for the deep review. I would like to resolve the char * issue when calling via Python, and then this should be ready. I think.

@@ -94,7 +94,7 @@
# there are a few files we purposely skip
files_to_skip = {"_1a-Long0.0.idf", "_ExternalInterface-actuator.idf", "_ExternalInterface-schedule.idf",
"_ExternalInterface-variable.idf", "HVAC3Zone-IntGains-Def.imf", "HVAC3ZoneChillerSpec.imf",
"HVAC3ZoneGeometry.imf", "HVAC3ZoneMat-Const.imf"}
"HVAC3ZoneGeometry.imf", "HVAC3ZoneMat-Const.imf", "_1ZoneUncontrolled_ForAPITesting.idf"}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made a special IDF that I can fiddle with all willy-nilly and not bother the core 1ZoneUncontrolled.idf file. But I need to ignore it here in custom_check.

@@ -991,7 +991,7 @@ if(BUILD_TESTING)
-DEPW_FILE=USA_CO_Golden-NREL.724666_TMY3.epw -P ${PROJECT_SOURCE_DIR}/cmake/RunCallbackTest.cmake)

set(EPW_FILE "${PROJECT_SOURCE_DIR}/weather/USA_IL_Chicago-OHare.Intl.AP.725300_TMY3.epw")
set(IDF_FILE "${PROJECT_SOURCE_DIR}/testfiles/1ZoneUncontrolled.idf")
set(IDF_FILE "${PROJECT_SOURCE_DIR}/testfiles/_1ZoneUncontrolled_ForAPITesting.idf")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use the specialized file for testing here, this way we can do things like add EMS vars to the IDF without affecting the core 1ZoneUncontrolled.idf file.

// In the API, we allow the user to manipulate user-defined EMS globals, but we skip the built-in ones to avoid
// problems. This constexpr should be incremented if we ever add more built-ins so that we skip the right amount
// in the API calls.
static int constexpr NumBuiltInErlVariables = 27;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, so create a single location constexpr where we declare the number of built-in EMS variables.

@@ -160,6 +160,9 @@ void InitializeRuntimeLanguage(EnergyPlusData &state)
state.dataRuntimeLangProcessor->ActualTimeNum = NewEMSVariable(state, "ACTUALTIME", 0);
state.dataRuntimeLangProcessor->WarmUpFlagNum = NewEMSVariable(state, "WARMUPFLAG", 0);

// this ensures we stay in sync with the number of skipped built-ins for API calls
assert(state.dataRuntimeLang->NumErlVariables == state.dataRuntimeLang->NumBuiltInErlVariables);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And here make sure we abide by that number of built-in variables. Which we do. Thanks for the heads up here @jmarrec

Comment on lines 243 to 249
char *inputFilePath(EnergyPlusState state)
{
const auto *thisState = static_cast<EnergyPlus::EnergyPlusData *>(state);
char *p = new char[std::strlen(thisState->dataStrGlobals->inputFilePath.string().c_str()) + 1];
std::strcpy(p, thisState->dataStrGlobals->inputFilePath.string().c_str());
return p;
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment on lines +128 to +129
/// \return A char * of the input file path. This allocates a new char *, and calling clients must free this when done with it!
ENERGYPLUSLIB_API char *inputFilePath(EnergyPlusState state);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes....but now I think I need to add something to the Python API. Once Python calls this function and converts it into a Python string (PyObject*), the C string is still hanging out. I may need a very minimal function in the C API that can free one of these char*. Thoughts @jmarrec ?

@@ -91,11 +91,27 @@ void afterZoneTimeStepHandler(EnergyPlusState state)
wallConstruction = getConstructionHandle(state, "R13WALL");
floorConstruction = getConstructionHandle(state, "FLOOR");

// don't forget to free memory from the API!
// checking for EMS globals
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test EMS global access via C API.

@@ -106,6 +122,13 @@ void afterZoneTimeStepHandler(EnergyPlusState state)
wallConstruction == -1 || floorConstruction == -1) {
exit(1);
}

char *idfPath = inputFilePath(state);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test IDF/EPW access via C API.

@@ -101,6 +101,26 @@ def time_step_handler(state):

wall_construction_handle = api.exchange.get_construction_handle(state, "R13WALL")
floor_construction_handle = api.exchange.get_construction_handle(state, "FLOOR")

idf_path = api.exchange.get_input_file_path(state)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test IDF/EPW access via Python API.

print(f"Got an input file path of: {idf_path}, and weather file path of: {epw_path}")

# checking for EMS globals
ems_global_valid = api.exchange.get_ems_global_handle(state, "MaximumEffort")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test EMS global access via Python API.

@Myoldmopar
Copy link
Member Author

Interesting, that number isn't always 27 at that point in the code @jmarrec. I may have to be a little more careful...

@Myoldmopar
Copy link
Member Author

This is completely green now that I'm doing proper index handling. If there are any issues, speak up, otherwise this will merge soon.

@Myoldmopar
Copy link
Member Author

Alright, dropping this in as well. It's all happy locally.

@Myoldmopar Myoldmopar merged commit 2fda843 into develop Aug 19, 2024
15 checks passed
@Myoldmopar Myoldmopar deleted the ACoupleAPIEndpoints branch August 19, 2024 17:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NewFeature Includes code to add a new feature to EnergyPlus
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants