Getting Started

This Getting_Started.md file was written as a quick-start guide to using TESP, specifically for DSO+T.

DSO+T at a Glance

There are two main parameters used in the DSO+T study that are managed by having separate configuration files depending on the case. The first is the number of nodes on the system. There is an 8-node case and a 200-node case. The 8-node case is a simplified version of the ERCOT system, and is assumed to be the default case due to its faster simulation time. The second is the DER scenario: which can be “business-as-usual” (BAU) or the high-renewables case. We assume a modified version of the high-renewables case from the DSO+T study, updated to align with present trends. The naming convention for DSO+T is [node]_[scenario]_... such that the case config file is 8_hi_system_case_config.json, for example.

Where you will be working:

Directory: TESP\examples\analysis\dsot\code

These are the main files to edit in order to generate new runs:

  • 8_hi_system_case_config.json

  • generate_case.py

  • prepare_case_dsot.py

Note that TESP expects python3, so any python scripts are run like:

python3 prepare_case_dsot.py

The main files behind tesp, running the market, the agents, the substations, are located within TESP/src/tesp_support/tesp_support/dsot

Before starting any runs, get necessary data:

Start by downloading supporting data that is not stored in the repository due to its size and static nature. This will add a “data” folder alongside the existing “code” folder from the repository. From the terminal:

cd tesp/examples/analysis/dsot/code
./dsotData.sh

Prepare a run

Edit the following files to setup your case:

  • 8_hi_system_case_config.json (Edit the following setup your case)

    • Market (true or false)

    • Start Time

    • End Time

    • Tmax (this is the alloted time based on your simulation window in seconds)

  • generate_case.py (For generating a year’s worth of runs, one month at a time)

    • Looks at prepare_case_dsot.py for most of its config

    • If attempting to generate only a selection of months, rather than the full year:

      if split_case:
    while True:
        for i in range(3,4): \\ Where months are (inclusive, exclusive)

    • NOTE: the naming convention for the runs generated by generate_case.py are [node]_[year]_[month]_[flag(s)] such that for the 8 node case with solar, battery, electric vehicles, and flexible loads available, the January case folder would be 8_2016_01_pv_bt_fl_ev.

  • prepare_case_dsot.py (For generating a run less than a month long, with duration set in 8_hi_system_case_config.json)

    • "RECS_data" = True

    • Set agent flags, pv, bt, ev, and fl to 1 or 0

    • Select whether to use 8 or 200 node test case

Select a Rate Design

Three example rate designs are available to apply to the system. Selecting a rate is part of the case prep. Those rates are: - The original transactive rate used for the DSO+T study - Flat rate - Time of Use

Each rate is described below:

Transactive (DSO+T) Rate

The default rate design uses the DSO+T rate, by setting "rate": "" within the system case config file. This enrolls all participating customers and devices in the transactive energy market. Set the DER percentages via the "dsoRECSAgentFile": "rates_analysis_case_config.json". This file lives in the dsot/data directory. For the rates scenario analysis, those parameters are:

"TransactiveHousePercentage": 80,
"SolarPercentage": 11,
"StoragePercentage": 3,
"EVPercentage": 8,

Where TransactiveHousePercentage defines the percentage of households that enroll all household DER devices in the transactive rate. This includes water heaters and programmable thermostats.

Flat Rate

The flat rate is the baseline case without a transactive market with a static price sent to all customers. To enable this rate, set "market": false in 8_hi_system_case_config.json.

Time of Use

The Time of Use, or TOU rate, uses a scheduled, time-varying price for electricity, with high and low pricing windows coincident with peak and off-peak pricing, respectively. To enable this rate, select "rate": "TOU" in 8_hi_system_case_config.json. To customize the TOU rate, see tou.py, in src\tesp_support\tesp_support\dsot.

TOU Options:

  • Seasons

  • Window timing

  • Peak/Off-Peak Ratio

  • Base price

  • Step Size (5 min or 1 hour)

The time_of_use_price_profile within tou.py creates a year-long profile as a .csv. That profile is read as tou_params to be used by the dso_rate_making.py.

Shell scripts to navigate runs (execute within the generated run folder):

Once your case folder(s) have been created, to start, stop, and clean up run files, the following commands can be run from the terminal [For windows users, in mobaxterm (rather than VSCode, as sometimes they don’t queue correctly)]. From the desired case folder:

  • ./run.sh : runs a run

  • ./kill.sh : kills a run

  • ./clean.sh : cleans up run files if you need to restart a killed run

Check on Run Status

From command line:

  • Check for errors (from the run folder):

cat *.log | grep -i err
cat */*.log | grep -i err

  • Check processes with htop.

    • If very little computing power is being used, likely no runs are active. This also shows each process. Can be sorted by user, time, CPU%, etc.

  • Check what is running with ps -a.

    • Used to make sure tesp install is successful. Occasionally GridLAB-D may not install correctly. If it is not listed after you’ve executed a run, try to reinstall it.

  • Check opf.csv and pf.csv file sizes.

    • Refresh case file directory and check size of opf.csv and pf.csv. These should be growing in size as things are written.

  • Check tso.log.

    • After run is completed, check the tso.log within the case file directory for any errors. Scroll to bottom to see that it finished successfully or exited with an error.

    • tail -20 tso.log can be used to see whether the run is still in progress, or has completed. The last few lines of the tso.log from a successful run look like:

      INFO:root:entering to_frame, filename=bus_8_2016_03_pv_bt_fl_ev_metrics.h5
INFO:root:entering to_frame, filename=gen_8_2016_03_pv_bt_fl_ev_metrics.h5
INFO:root:entering to_frame, filename=sys_8_2016_03_pv_bt_fl_ev_metrics.h5
INFO:root:entering to_frame, filename=da_lmp_8_2016_03_pv_bt_fl_ev_metrics.h5
INFO:root:entering to_frame, filename=da_line_8_2016_03_pv_bt_fl_ev_metrics.h5
INFO:root:entering to_frame, filename=da_gen_q_8_2016_03_pv_bt_fl_ev_metrics.h5
INFO:root:entering to_frame, filename=da_q_8_2016_03_pv_bt_fl_ev_metrics.h5
INFO:root:entering to_frame, filename=rt_line_8_2016_03_pv_bt_fl_ev_metrics.h5
INFO:root:entering to_frame, filename=rt_q_8_2016_03_pv_bt_fl_ev_metrics.h5
INFO:root:breaking out at 2764800
INFO:root:finalize metrics writing
INFO:root:closing files
INFO:root:finalizing HELICS tso federate

Troubleshooting Runs

  • Address already in use:

    • Are you already running something? Make sure it’s finished, i.e., don’t try to postprocess and run something new at the same time.

  • Infeasible solution/ No RT starting point:

    • genPowerLevel needs to be adjusted in 8_hi_system_case_config.json

      • Defines the initial power output for generators when running the very first timestep. This allows them to be put in such a state that, when respecting ramp rates, they can reach a reasonable dispatch.

      • 0.6 - 0.7 usually works, for high-demand months, might need to go up to 0.85. Low-demand months might need to go to 0.45.

  • Prepare case or generate case fails:

    • Did you update RECS parameters? If so, remember to re-run recs_gld_house_parameters.py

Run Post-Processing

Monthly Case Post-Processing

At the end of each simulation (monthly run) automated post-processing is run on the results: run_case_postprocessing.py. The purpose of this initial analysis is to generate summary data and initial review plots (saved in the /plots folder within the month case folder) so not all data needs to be transferred prior to the annual analysis and review. This step enables parallel processing.

Post-processing can be run either through the shell script, ./postprocesss.sh or with the command:

python3 ../run_case_postprocessing.py > postprocessing.log&

Annual Case Post-Processing

Once the monthly post-processing is completed on all (e.g., 12) monthly cases for a scenario the annual post processing can be run: run_annual_postprocessing.py

This is a semi-automated process that compiles annual summary data of key metrics (e.g., loads, LMPs) and analysis (e.g. customer billing, customer cash flow sheets, and DSO cash flow sheets). In its current state this process has to be run twice to iterate on expenses and revenues (to be fixed).

NOTE: This step also tunes the DSO agent LMP forecaster based on the annual DA LMPs. The resulting files will then need to be provided back for future simulations. For example, the annual post-processing results for a baseline case (e.g., BAU or Flat rate) should be used as the tuned LMP forecaster for later transactive cases.

Case Comparison and Reporting Exhibit Generation

At the end of the annual post-processing for each case (e.g., Flat, TOU, DSOT) comparison plots and tables can be generated for results review and reporting. The plotting typically relies on plots.py and case_comparison_plots.py and can be customized based on the scenarios, months, days, and variables compared.

Typically, tables are created by pasting the results generated in .csv files (e.g., DSO_loads_stats.csv or DSO_CSF_Summary.csv) into excel workbooks to create bespoke table formats.

The example script plotting.py shows example use of the plotting functions to generate useful visual outputs. Generally, plots are saved in the plots folder of the associated target scenario or month, but plots folders will/can exist at the case comparison, case, and month levels.

Results Validation and Verification Checks

  • Check identical customer populations across cases (for example, in RCI.csv and Master_Customer_Dataframe.csv).

  • Check identical date ranges and time periods of evaluation between cases (for example in DSO_Total_Loads.csv).

  • Check that DSO revenues equal expenses (capital and operating) within cases in DSO_CFS_summary.csv.

Move run files over to a sharefolder (FOR PNNL USERS ONLY):

If you would like to share your results with your team or access the run files from your PC:

  1. Navigate to the folder one directory above the run folder you would like to move

  2. sudo mount -t cifs //pnnlfs09.pnl.gov/sharedata37_op$/DSOT  /mnt/dsot -o username=[USER]

  3. sudo cp -r [run_folder] /mnt/dsot/run_outputs/Rates_Scenario/.

Note instructions will change based on mount location, folder, and target directory.

If choosing to delete run folders to clear up room, do so from the mobaxterm terminal rather than VSCode to ensure they are cleared from the disk.