Skip to content

Commit 9848130

Browse files
authored
Merge pull request #1008 from hlindberg/MODULES-8760_support-iterative-build-in-merge-function
(MODULES-8760) Add iterative feature to merge() function
2 parents b0dd4c1 + af96187 commit 9848130

File tree

3 files changed

+181
-7
lines changed

3 files changed

+181
-7
lines changed

README.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1986,6 +1986,33 @@ Since Puppet 4.0.0, you can use the + operator to achieve the same merge.
19861986
19871987
$merged_hash = $hash1 + $hash2
19881988
1989+
If merge is given a single `Iterable` (`Array`, `Hash`, etc.) it will call a given block with
1990+
up to three parameters, and merge each resulting Hash into the accumulated result. All other types
1991+
of values returned from the block (typically `undef`) are skipped (not merged).
1992+
1993+
The codeblock can take 2 or three parameters:
1994+
* with two, it gets the current hash (as built to this point), and each value (for hash the value is a [key, value] tuple)
1995+
* with three, it gets the current hash (as built to this point), the key/index of each value, and then the value
1996+
1997+
If the iterable is empty, or no hash was returned from the given block, an empty hash is returned. In the given block, a call to `next()`
1998+
will skip that entry, and a call to `break()` will end the iteration.
1999+
2000+
*Example: counting occurrences of strings in an array*
2001+
```puppet
2002+
['a', 'b', 'c', 'c', 'd', 'b'].merge | $hsh, $v | { { $v => $hsh[$v].lest || { 0 } + 1 } }
2003+
# would result in { a => 1, b => 2, c => 2, d => 1 }
2004+
```
2005+
2006+
*Example: skipping values for entries that are longer than 1 char*
2007+
2008+
```puppet
2009+
['a', 'b', 'c', 'c', 'd', 'b', 'blah', 'blah'].merge | $hsh, $v | { if $v =~ String[1,1] { { $v => $hsh[$v].lest || { 0 } + 1 } } }
2010+
# would result in { a => 1, b => 2, c => 2, d => 1 } since 'blah' is longer than 2 chars
2011+
```
2012+
2013+
The iterative `merge()` has an advantage over doing the same with a general `reduce()` in that the constructed hash
2014+
does not have to be copied in each iteration and thus will perform much better with large inputs.
2015+
19892016
*Type*: rvalue.
19902017
19912018
#### `min`

lib/puppet/functions/merge.rb

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# Merges two or more hashes together or hashes resulting from iteration, and returns the resulting hash.
2+
#
3+
# @example Using merge()
4+
#
5+
# $hash1 = {'one' => 1, 'two', => 2}
6+
# $hash2 = {'two' => 'dos', 'three', => 'tres'}
7+
# $merged_hash = merge($hash1, $hash2)
8+
# # The resulting hash is equivalent to:
9+
# # $merged_hash = {'one' => 1, 'two' => 'dos', 'three' => 'tres'}
10+
#
11+
# When there is a duplicate key, the key in the rightmost hash will "win."
12+
#
13+
# Note that since Puppet 4.0.0 the same merge can be achieved with the + operator.
14+
#
15+
# $merged_hash = $hash1 + $hash2
16+
#
17+
# If merge is given a single Iterable (Array, Hash, etc.) it will call a given block with
18+
# up to three parameters, and merge each resulting Hash into the accumulated result. All other types
19+
# of values returned from the block (typically undef) are skipped (not merged).
20+
#
21+
# The codeblock can take 2 or three parameters:
22+
# * with two, it gets the current hash (as built to this point), and each value (for hash the value is a [key, value] tuple)
23+
# * with three, it gets the current hash (as built to this point), the key/index of each value, and then the value
24+
#
25+
# If the iterable is empty, or no hash was returned from the given block, an empty hash is returned. In the given block, a call to `next()`
26+
# will skip that entry, and a call to `break()` will end the iteration.
27+
#
28+
# @example counting occurrences of strings in an array
29+
# ['a', 'b', 'c', 'c', 'd', 'b'].merge | $hsh, $v | { { $v => $hsh[$v].lest || { 0 } + 1 } }
30+
# # would result in { a => 1, b => 2, c => 2, d => 1 }
31+
#
32+
# @example skipping values for entries that are longer than 1 char
33+
# ['a', 'b', 'c', 'c', 'd', 'b', 'blah', 'blah'].merge | $hsh, $v | { if $v =~ String[1,1] { { $v => $hsh[$v].lest || { 0 } + 1 } } }
34+
# # would result in { a => 1, b => 2, c => 2, d => 1 } since 'blah' is longer than 2 chars
35+
#
36+
# The iterative `merge()` has an advantage over doing the same with a general `reduce()` in that the constructed hash
37+
# does not have to be copied in each iteration and thus will perform much better with large inputs.
38+
#
39+
Puppet::Functions.create_function(:merge) do
40+
dispatch :merge2hashes do
41+
repeated_param 'Variant[Hash, Undef, String[0,0]]', :args # this strange type is backwards compatible
42+
return_type 'Hash'
43+
end
44+
45+
dispatch :merge_iterable3 do
46+
repeated_param 'Iterable', :args
47+
block_param 'Callable[3,3]', :block
48+
return_type 'Hash'
49+
end
50+
51+
dispatch :merge_iterable2 do
52+
repeated_param 'Iterable', :args
53+
block_param 'Callable[2,2]', :block
54+
return_type 'Hash'
55+
end
56+
57+
def merge2hashes(*hashes)
58+
accumulator = {}
59+
hashes.each { |h| accumulator.merge!(h) if h.is_a?(Hash) }
60+
accumulator
61+
end
62+
63+
def merge_iterable2(iterable)
64+
accumulator = {}
65+
enum = Puppet::Pops::Types::Iterable.asserted_iterable(self, iterable)
66+
enum.each do |v|
67+
r = yield(accumulator, v)
68+
accumulator.merge!(r) if r.is_a?(Hash)
69+
end
70+
accumulator
71+
end
72+
73+
def merge_iterable3(iterable)
74+
accumulator = {}
75+
enum = Puppet::Pops::Types::Iterable.asserted_iterable(self, iterable)
76+
if enum.hash_style?
77+
enum.each do |entry|
78+
r = yield(accumulator, *entry)
79+
accumulator.merge!(r) if r.is_a?(Hash)
80+
end
81+
else
82+
begin
83+
index = 0
84+
loop do
85+
r = yield(accumulator, index, enum.next)
86+
accumulator.merge!(r) if r.is_a?(Hash)
87+
index += 1
88+
end
89+
rescue StopIteration # rubocop:disable Lint/HandleExceptions
90+
end
91+
end
92+
accumulator
93+
end
94+
end

spec/functions/merge_spec.rb

Lines changed: 60 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2,20 +2,34 @@
22

33
describe 'merge' do
44
it { is_expected.not_to eq(nil) }
5-
it { is_expected.to run.with_params.and_raise_error(Puppet::ParseError, %r{wrong number of arguments}i) }
6-
it { is_expected.to run.with_params({}, 'two').and_raise_error(Puppet::ParseError, %r{unexpected argument type String}) }
7-
it { is_expected.to run.with_params({}, 1).and_raise_error(Puppet::ParseError, %r{unexpected argument type (Fixnum|Integer)}) }
8-
it { is_expected.to run.with_params({ 'one' => 1, 'three' => { 'four' => 4 } }, 'two' => 'dos', 'three' => { 'five' => 5 }).and_return('one' => 1, 'three' => { 'five' => 5 }, 'two' => 'dos') }
9-
105
it {
11-
pending 'should not special case this'
12-
is_expected.to run.with_params({}).and_return({})
6+
is_expected.to run \
7+
.with_params({}, 'two') \
8+
.and_raise_error(
9+
ArgumentError, \
10+
Regexp.new(Regexp.escape("rejected: parameter 'args' expects a value of type Undef, Hash, or String[0, 0], got String")),
11+
)
12+
}
13+
it {
14+
is_expected.to run \
15+
.with_params({}, 1) \
16+
.and_raise_error(ArgumentError, %r{parameter 'args' expects a value of type Undef, Hash, or String, got Integer})
17+
}
18+
it {
19+
is_expected.to run \
20+
.with_params({ 'one' => 1, 'three' => { 'four' => 4 } }, 'two' => 'dos', 'three' => { 'five' => 5 }) \
21+
.and_return('one' => 1, 'three' => { 'five' => 5 }, 'two' => 'dos')
1322
}
23+
24+
it { is_expected.to run.with_params.and_return({}) }
25+
it { is_expected.to run.with_params({}).and_return({}) }
1426
it { is_expected.to run.with_params({}, {}).and_return({}) }
1527
it { is_expected.to run.with_params({}, {}, {}).and_return({}) }
28+
1629
describe 'should accept empty strings as puppet undef' do
1730
it { is_expected.to run.with_params({}, '').and_return({}) }
1831
end
32+
1933
it { is_expected.to run.with_params({ 'key' => 'value' }, {}).and_return('key' => 'value') }
2034
it { is_expected.to run.with_params({}, 'key' => 'value').and_return('key' => 'value') }
2135
it { is_expected.to run.with_params({ 'key' => 'value1' }, 'key' => 'value2').and_return('key' => 'value2') }
@@ -24,4 +38,43 @@
2438
.with_params({ 'key1' => 'value1' }, { 'key2' => 'value2' }, 'key3' => 'value3') \
2539
.and_return('key1' => 'value1', 'key2' => 'value2', 'key3' => 'value3')
2640
}
41+
describe 'should accept iterable and merge produced hashes' do
42+
it {
43+
is_expected.to run \
44+
.with_params([1, 2, 3]) \
45+
.with_lambda { |_hsh, val| { val => val } } \
46+
.and_return(1 => 1, 2 => 2, 3 => 3)
47+
}
48+
it {
49+
is_expected.to run \
50+
.with_params([1, 2, 3]) \
51+
.with_lambda { |_hsh, val| { val => val } unless val == 2 } \
52+
.and_return(1 => 1, 3 => 3)
53+
}
54+
it {
55+
is_expected.to run \
56+
.with_params([1, 2, 3]) \
57+
# rubocop:disable Style/Semicolon
58+
.with_lambda { |_hsh, val| raise StopIteration if val == 3; { val => val } } \
59+
.and_return(1 => 1, 2 => 2)
60+
}
61+
it {
62+
is_expected.to run \
63+
.with_params(['a', 'b', 'b', 'c', 'b']) \
64+
.with_lambda { |hsh, val| { val => (hsh[val] || 0) + 1 } } \
65+
.and_return('a' => 1, 'b' => 3, 'c' => 1)
66+
}
67+
it {
68+
is_expected.to run \
69+
.with_params(['a', 'b', 'c']) \
70+
.with_lambda { |_hsh, idx, val| { idx => val } } \
71+
.and_return(0 => 'a', 1 => 'b', 2 => 'c')
72+
}
73+
it {
74+
is_expected.to run \
75+
.with_params('a' => 'A', 'b' => 'B', 'c' => 'C') \
76+
.with_lambda { |_hsh, key, val| { key => "#{key}#{val}" } } \
77+
.and_return('a' => 'aA', 'b' => 'bB', 'c' => 'cC')
78+
}
79+
end
2780
end

0 commit comments

Comments
 (0)