Skip to content

CLN: ASV sparse #19047

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jan 3, 2018
Merged

CLN: ASV sparse #19047

merged 1 commit into from
Jan 3, 2018

Conversation

mroeschke
Copy link
Member

Simplified and created a top level function that made sparse arrays (which was repeated throughout the file). Otherwise, simplified benchmarks with param where possible.

$ asv dev -b ^sparse
· Discovering benchmarks
· Running 15 total benchmarks (1 commits * 1 environments * 15 benchmarks)
[  0.00%] ·· Building for existing-py_home_matt_anaconda_envs_pandas_dev_bin_python
[  0.00%] ·· Benchmarking existing-py_home_matt_anaconda_envs_pandas_dev_bin_python
[  6.67%] ··· Running sparse.Arithmetic.time_add                             ok
[  6.67%] ···· 
               ================== ======== ========
               --                     fill_value   
               ------------------ -----------------
                dense_proportion     0       nan   
               ================== ======== ========
                      0.1          85.6ms   68.0ms 
                      0.01         8.46ms   67.5ms 
               ================== ======== ========

[ 13.33%] ··· Running sparse.Arithmetic.time_divide                          ok
[ 13.33%] ···· 
               ================== ======== ========
               --                     fill_value   
               ------------------ -----------------
                dense_proportion     0       nan   
               ================== ======== ========
                      0.1          82.7ms   67.6ms 
                      0.01         8.94ms   67.6ms 
               ================== ======== ========

[ 20.00%] ··· Running sparse.Arithmetic.time_intersect                       ok
[ 20.00%] ···· 
               ================== ======== =======
               --                    fill_value   
               ------------------ ----------------
                dense_proportion     0       nan  
               ================== ======== =======
                      0.1          5.72ms   344ms 
                      0.01         356μs    343ms 
               ================== ======== =======

[ 26.67%] ··· Running sparse.Arithmetic.time_make_union                      ok
[ 26.67%] ···· 
               ================== ======== =======
               --                    fill_value   
               ------------------ ----------------
                dense_proportion     0       nan  
               ================== ======== =======
                      0.1          79.4ms   715ms 
                      0.01         8.63ms   742ms 
               ================== ======== =======

[ 33.33%] ··· Running sparse.ArithmeticBlock.time_addition                   ok
[ 33.33%] ···· 
               ============ ========
                fill_value          
               ------------ --------
                   nan       8.20ms 
                    0        8.03ms 
               ============ ========

[ 40.00%] ··· Running sparse.ArithmeticBlock.time_division                   ok
[ 40.00%] ···· 
               ============ ========
                fill_value          
               ------------ --------
                   nan       8.39ms 
                    0        8.04ms 
               ============ ========

[ 46.67%] ··· Running sparse.ArithmeticBlock.time_intersect                  ok
[ 46.67%] ···· 
               ============ ========
                fill_value          
               ------------ --------
                   nan       3.51ms 
                    0        3.69ms 
               ============ ========

[ 53.33%] ··· Running sparse.ArithmeticBlock.time_make_union                 ok
[ 53.33%] ···· 
               ============ ========
                fill_value          
               ------------ --------
                   nan       7.86ms 
                    0        8.19ms 
               ============ ========

[ 60.00%] ··· Running sparse.FromCoo.time_sparse_series_from_coo         3.30ms
[ 66.67%] ··· Running ...se.SparseArrayConstructor.time_sparse_array         ok
[ 66.67%] ···· 
               ================== ============ ======================== ========
                dense_proportion   fill_value           dtype                   
               ------------------ ------------ ------------------------ --------
                      0.1              0         <type 'numpy.int64'>    46.7ms 
                      0.1              0        <type 'numpy.float64'>   47.1ms 
                      0.1              0           <type 'object'>       104ms  
                      0.1             nan        <type 'numpy.int64'>    364ms  
                      0.1             nan       <type 'numpy.float64'>   47.7ms 
                      0.1             nan          <type 'object'>       116ms  
                      0.01             0         <type 'numpy.int64'>    8.80ms 
                      0.01             0        <type 'numpy.float64'>   8.71ms 
                      0.01             0           <type 'object'>       67.5ms 
                      0.01            nan        <type 'numpy.int64'>    380ms  
                      0.01            nan       <type 'numpy.float64'>   9.57ms 
                      0.01            nan          <type 'object'>       64.7ms 
               ================== ============ ======================== ========

[ 73.33%] ··· Running ...SparseDataFrameConstructor.time_constructor      6.39s
[ 80.00%] ··· Running ...e.SparseDataFrameConstructor.time_from_dict      245ms
[ 86.67%] ··· Running ....SparseDataFrameConstructor.time_from_scipy      563ms
[ 93.33%] ··· Running ...se.SparseSeriesToFrame.time_series_to_frame      280ms
[100.00%] ··· Running sparse.ToCoo.time_sparse_series_to_coo             43.9ms

@jreback jreback added Benchmark Performance (ASV) benchmarks Sparse Sparse Data Type labels Jan 3, 2018
@jreback jreback merged commit e581b14 into pandas-dev:master Jan 3, 2018
@jreback
Copy link
Contributor

jreback commented Jan 3, 2018

thanks!

@jreback jreback added this to the 0.23.0 milestone Jan 3, 2018
@mroeschke mroeschke deleted the asv_clean_sparse branch January 3, 2018 17:39
jbrockmendel
jbrockmendel reviewed Feb 19, 2020
View reviewed changes
asv_bench/benchmarks/sparse.py
replace=False)
for ind in indicies:
arr[ind:ind + block_size] = np.random.randint(0, 100, block_size)
return SparseArray(arr, fill_value=fill_value)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mroeschke looking at this now, can we turn the Arithmetic class above into a special case of ArithmeticBlock?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure I suppose so, though I didn't write the original benchmarks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jbrockmendel jbrockmendel jbrockmendel left review comments

Assignees
No one assigned
Labels
Benchmark Performance (ASV) benchmarks Sparse Sparse Data Type
Projects
None yet
Milestone
0.23.0
Development

Successfully merging this pull request may close these issues.
3 participants
@mroeschke @jreback @jbrockmendel