add skim data like tour and trip distance to output files

[bstabler]

Add fields to the output tables including key travel data such as time, distance, and cost via tour and trip annotation expressions

[bettinardi]

I'm hoping for other aspects / detail that occur with in the model as well, as optional outputs. For me I think of what a survey looks like, and then an augmented survey record. So let's think about all the fields that are on a household survey and whether or not the ABM can provide them. And similarly, let's think about all the fields that would exist on an augmented survey. Specifically I'm thinking about the choice the user made, but also the other choices that were available to them and the costs and distances for those choices as well.

Let's make it really easy for the user to specify any of the interim fields that might get appended to various tables at different points in the model run.

[bstabler]

Task Spec

The existing annotation functionality is for adding additional data columns to pipeline tables (and output tables if written out). We've been adding the ability to annotate tables before and after each model step as needed. An existing example is the persons workplace location annotation expression file which adds many fields to the persons table, including distance_to_work based on the person home taz, work taz, and the distance skim.

We'll add this feature to the tour and trip destination choice and mode choice models so users can write custom expressions to add additional output fields such as tour and trip distance. Here's an example of additional settings to add to the atwork_subtour_destination model so users can join skim data to the table. The atwork_subtour_destination.csv expression file will look similar to the persons workplace location annotation expression file above (but with less expressions).

postprocessor:
  SPEC: atwork_subtour_destination
  DF: tours
  TABLES:
    - skim_dict

We'll also create annotation expression files to exercise this functionality and update the user documentation to describe the new files and additional data fields written out in the example.

[bstabler]

@bettinardi - I think the table annotation functionality is a good fit for this need. Appending all the non-chosen alternatives data is also important and we're planning to build this functionality in the upcoming "estimation data bundle" task. The basic idea of the estimation data bundle task is to read an input household travel survey in activitysim format, join all the necessary choice data in order to estimate the models outside of activitysim, and write out all the tables as an "estimation data bundle".

[bettinardi]

This all sounds fine. Just noting that my end desired deliverable for this task is not just the functionality with some examples, but also having an example worked up that creates an activitySim default (standard) output table with tables that produce as many fields from household surveys as we can. Then ideally a second example expression default file that produces an augmented survey output. So the functionality all sounds right, and I like the idea of examples. I just want to make sure that there is a large/complete example that provides the partners with at least two flavors of common outputs that they would likely want to use (a lighter output of just survey fields, and a heavier output of survey + augmentation fields). Then there would be instructions for users to take the expression files and further customize beyond the provided defaults as desired.

[gregorkschroeder]

"key travel data such as time, distance, and cost"

At SANDAG this key travel data is appended to each trip record segmented by mode used on the trip. For example, an inbound Park and Ride to Transit trip has time/dist/cost for the auto access portion of the trip, time/dist/cost for the transit portion, and time/dist/cost for the walk egress portion. This level of granularity for trips is integral to SANDAG's usage of their ABM model. Is/Will this data be accessible?

[bstabler]

Yes, the goal of the functionality is to be able to append the relevant data you need. When the time is right, we'll work through some good examples for a multiple zone system model like SANDAG.

[bstabler]

in #325 we added functionality to add additional fields to the output trips table when building the final trip matrices