Improve test stability #4
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* limit multimethod version to 2.0 and earlier * add multimethod version to other settings * [makedocs] update installer download link * [makedocs] update branch docs
* use libmamba solver * add permissions [makedocs] * add write permission for dev docs [makedocs] * conda-solver: classic
…vitySim#901) Co-authored-by: Jeffrey Newman <[email protected]>
* use libmamba solver * add permissions [makedocs] * add write permission for dev docs [makedocs] * conda-solver: classic * include workflow dispatch option for tests * update release instructions * add installer build to instructions * Pin mamba for now, per conda-incubator/setup-miniconda#392 * conda-remove-defaults
…on_enhancements # Conflicts: # conda-environments/activitysim-dev-base.yml # conda-environments/activitysim-dev.yml # conda-environments/docbuild.yml # conda-environments/github-actions-tests.yml # pyproject.toml
* fix link * allow failure to import larch * workflow * blacken * try some pins * speed up docbuild * use pandas 2 for docs * oops wrong file * restore foundation
There was a problem hiding this comment.
Pull Request Overview
This PR improves test stability for the estimation module by updating model parameter constraints, modifying test cases, and adjusting version checks. Key changes include adjusting parameter locking in the trip destination model tests, updating larch version handling to correctly process development versions, and updating documentation and CI configuration files.
Reviewed Changes
Copilot reviewed 17 out of 23 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| docs/users-guide/estimation-mode/general.md | Added documentation for using Larch and fixed a typo in the parameter name for optimization. |
| conda-environments/* | Updated dependencies by adding a multimethod version constraint. |
| activitysim/estimation/test/test_larch_estimation.py | Adjusted test cases to use the SLSQP algorithm and applied a parameter lock for the trip destination model. |
| activitysim/estimation/larch/location_choice.py | Improved handling of alternative availability by adding an optional availability expression parameter. |
| activitysim/estimation/larch/init.py | Modified version check logic to account for development versions of larch. |
| activitysim/core/workflow/tracing.py | Revised table selection for tracing by preferring proto tables when available. |
| activitysim/abm/models/disaggregate_accessibility.py | Changed deregistration of traceable tables to drop all proto-related tables. |
| HOW_TO_RELEASE.md | Updated release instructions including tagging and environment handling commands. |
| .github/workflows/* | Updated GitHub Actions configuration with new setup options and tooling versions. |
Files not reviewed (6)
- activitysim/estimation/test/test_larch_estimation/test_loc_trip_destination_BHHH_loglike.csv: Language not supported
- activitysim/estimation/test/test_larch_estimation/test_loc_trip_destination_SLSQP_loglike.csv: Language not supported
- activitysim/estimation/test/test_larch_estimation/test_loc_trip_destination_SLSQP_size_spec.csv: Language not supported
- activitysim/estimation/test/test_larch_estimation/test_location_model_trip_destination_BHHH_None_.csv: Language not supported
- activitysim/estimation/test/test_larch_estimation/test_location_model_trip_destination_SLSQP_None_.csv: Language not supported
- docs/gettingstarted.rst: Language not supported
Comments suppressed due to low confidence (1)
docs/users-guide/estimation-mode/general.md:31
- Typo found: 'mathod' should be corrected to 'method' to accurately reflect the intended argument name.
The `larch.Model.estimate` method allows the user to specify the optimization algorithm to use via the `mathod` argument, which can be set to 'BHHH', 'SLSQP', or any other algorithm ...
| generate a new tag with the version number "1.2.3" (for example): | ||
| ```sh | ||
| bump2version patch --new-version 1.2.3 --list | ||
| git -a v1.2.3 -m "Release v1.2.3" |
There was a problem hiding this comment.
The git command for tagging a release appears to be incorrect. It should likely use 'git tag -a v1.2.3 -m "Release v1.2.3"' instead of 'git -a v1.2.3 -m "Release v1.2.3"'.
Suggested change
| git -a v1.2.3 -m "Release v1.2.3" | |
| git tag -a v1.2.3 -m "Release v1.2.3" |
…on_enhancements # Conflicts: # activitysim/estimation/larch/__init__.py # activitysim/estimation/larch/cdap.py # activitysim/estimation/larch/general.py # activitysim/estimation/larch/location_choice.py # activitysim/estimation/larch/mode_choice.py # activitysim/estimation/larch/nonmand_tour_freq.py # activitysim/estimation/larch/scheduling.py # activitysim/estimation/larch/simple_simulate.py # activitysim/estimation/larch/stop_frequency.py
This reverts commit be5d093.
dhensle
approved these changes
May 15, 2025
dhensle
added a commit
that referenced
this pull request
Oct 2, 2025
* multiprocess initial commit * blacken * parquet format for EDBs * adding pkl, fixing edb concat and write * fixing double naming of coefficient files * blacken * fixing missing cdap coefficients file, write pickle function * combact edb writing, index duplication, parquet datatypes * sorting dest choice bundles * adding coalesce edbs as its own step * CI testing initial commit * infer.py CI testing * estimation sampling for non-mandatory and joint tours * adding survey choice to choices_df in interaction_sample * adding option to delete the mp edb subdirs * changes supporting sandag abm3 estimation mode * running test sandag example through trip dest sample * Estimation Pydantic (#2) * pydantic for estimation settings * allow df as type in config * fix table_info * repair for Pydantic * df is attribute * Estimation settings pydantic update * new compact formatting * handling multiple columns for parquet write * dropping duplicate columns * actually removing duplicate columns * dfs with correct indexes and correct mp sorting * ignore index on sort for mp coalesce edbs * updating estimation checks to allow for non-zero household_sample_size * Re-estimation (#3) * pydantic for estimation settings * allow df as type in config * fix table_info * auto ownership * repair for pydantic * update for ruff * updated for simple models * repair for Pydantic * simple simulate and location choice * df is attribute * scheduling * stop freq * test locations * cdap * nonmand_and_joint_tour_dest_choice * nonmand_tour_freq * fix ci to stop using mamba * test updates * use larch6 from pip * use numba for stop freq * fix for pandas 1.5 * fix stop freq test for numba * Sharrow Cache Dir Setting (ActivitySim#893) * setting necessary filesystem changes from settings file * set for multiprocessing * repair github actions * github action updates (ActivitySim#903) * script to make data * unified script for making data * remove older * bug * doc note * load from parquet if available * add original alt ids to EDB output when using compact * fix MP race * script arg to skip to EDB * clean up CDAP and blacken * refactor model_estimation_table_types change to estimation_table_types, to avoid pydantic namespace clash * repair drop_dupes * blacken * location choice with compact * choice_def for compact * spec changes for simple-simulate * re-estimation demo for auto ownership * clean up status messages * change name to stop pydantic warnings * edit configs * default estimation sample size is same as regular sample size * allow location alts not in cv format * dummy zones for location choice * update scheduling model estimation * various cleanup * stop freq * tidy build script * update 02 school location for larger example * update notebook 04 * editable model re-estimation for location choice * fix test names * update notebooks * cdap print filenames as loading * notebook 07 * tests thru 07 * notebooks 08 09 * build the data first * runnable script * change larch version dependency * keep pandas<2 * notebooks 10 11 * notebook 12 * remove odd print * add matplotlib * notebook 13 14 * test all the notebooks * add xlsxwriter to tests * notebook 15 * CDAP revise model spec demo * notebook 16 * notebook 17 * longer timeout * notebook 18 * notebook 19 * notebook 20 * smaller notebook 15 * configurable est mode setup * notebook 21 * notebook 22 * config sample size in GA * notebook 23 * updates for larch and graphviz * change default to compact * compare model 03 * test updates * rename test targets * repair_av_zq * move doctor up * add another repair * oops --------- Co-authored-by: David Hensle <[email protected]> * Removing estimation.yaml settings that are no longer needed * fixing unit tests, setting parquet edb default * one more missed estimation.yaml * using df.items for pandas 2 compatibility * tidy doc * updating edb file name for NMTF * updating numba and pandas in the conda env files * Improve test stability (#4) * handle dev versions of Larch * test stability * pin multimethod < 2.0 * add availability_expression * starting est docs * Resolve package version conflicts (ActivitySim#923) * limit multimethod version to 2.0 and earlier * add multimethod version to other settings * [makedocs] update installer download link * [makedocs] update branch docs * GitHub Actions updates (ActivitySim#926) * use libmamba solver * add permissions [makedocs] * add write permission for dev docs [makedocs] * conda-solver: classic * trace proto tables if available, otherwise synthetic population (ActivitySim#901) Co-authored-by: Jeffrey Newman <[email protected]> * release instructions (ActivitySim#927) * use libmamba solver * add permissions [makedocs] * add write permission for dev docs [makedocs] * conda-solver: classic * include workflow dispatch option for tests * update release instructions * add installer build to instructions * Pin mamba for now, per conda-incubator/setup-miniconda#392 * conda-remove-defaults * when no unavailability parameters are included * some general estimation docs * Use pandas 2 for docbuild environment (ActivitySim#928) * fix link * allow failure to import larch * workflow * blacken * try some pins * speed up docbuild * use pandas 2 for docs * oops wrong file * restore foundation * Update HOW_TO_RELEASE.md * refactor(shadow_pricing.py): remove a duplicated `default_segment_to_name_dict` (ActivitySim#930) * fix typo * fixing disaggregate accessibility bug in zone sampler * Revert "fixing disaggregate accessibility bug in zone sampler" This reverts commit be5d093. * notes on size terms * clean up docbuild * fix version check * add some doc * tidy * estimation docs * more on alternative avail * model evaluation * add doc on component_model * documentation enhancements * larch6 is now larch>6 * branch docs on workflow_dispatch * missing doc section on model respec --------- Co-authored-by: Yue Shuai <[email protected]> Co-authored-by: David Hensle <[email protected]> Co-authored-by: amarin <[email protected]> Co-authored-by: Ali Etezady <[email protected]> Co-authored-by: Sijia Wang <[email protected]> * handling missing data or availability conditions * add docs on locking size terms * include constants in CDAP * bump larch requirement * require larch 6.0.40 * add xlsxwriter to envs * require larch 6.0.41 * add links * fix typos and formatting * cdap hh and per parquet read match csv * add missing x_validator for mode choice and nonmand tour freq * add tour mode choice edit example * add to docs * union not addition on sets * restore nb kernel * blacken * replacing conda with uv in estimation tests * add requests to github-action dependencies * running with created virtual env instead * Fix estimation notebook tests (#8) * Update scheduling.py --------- Co-authored-by: Jeffrey Newman <[email protected]> Co-authored-by: Yue Shuai <[email protected]> Co-authored-by: amarin <[email protected]> Co-authored-by: Ali Etezady <[email protected]> Co-authored-by: Sijia Wang <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request includes several changes to the
activitysim/estimationmodule, primarily focusing on updating thelarchversion handling, modifying test cases, and adjusting model parameters.The key change is applying a constraint on the trip model choice model, to solve an over specification problem and ensure the estimation results are stable.
The most important changes are grouped by their themes below.
Test Case Adjustments:
test_location_modelfunction to lock a parameter and set a cap for the "trip_destination" model to ensure model identifiability.test_simple_simulatefunction inactivitysim/estimation/test/test_larch_estimation.pyto also use the "SLSQP" method for the "trip_destination" model. The SLSQP algorithm obeys constraints on parameter values, while the BHHH algorithm does not, yielding very different answers. Which is "better" depends on the modeler's opinion on the validity of the constraints.Version Handling:
larchversion check inactivitysim/estimation/larch/__init__.pyto handle development versions correctly.Model Parameter Updates:
test_loc_trip_destination_BHHH_loglike.csvandtest_loc_trip_destination_SLSQP_loglike.csvto reflect new model results. [1] [2]test_loc_trip_destination_SLSQP_size_spec.csv.CSV File Updates:
test_location_model_trip_destination_BHHH_None_.csvto reflect new model results.test_location_model_trip_destination_SLSQP_None_.csvwith updated parameter values for the "trip_destination" model.