Reduce memory by multiple GBs with many classic queues

[ansd]

Store directory names as binary instead of string.

This commit saves >1GB of memory per 100,000 classic queues v2. With longish node names, the memory savings are even much higher.

This commit is especially a prerequisite for scalable MQTT where every subscribing MQTT connection creates its own classic queue.

So, with 3 million MQTT subscribers, this commit saves >30GB of memory.

This commits stores file names as binaries and converts back to file:filename() when passed to file API functions. This is to reduce risk of breaking behaviour for path names containing unicode chars on certain platforms.

Prior to this commit, a classic queue process state contained many lists:

              {passthrough,rabbit_variable_queue,
                           {vqstate,{0,[],[]},
                                    {0,[],[]},
                                    {delta,undefined,0,0,undefined},
                                    {0,[],[]},
                                    {0,[],[]},
                                    0,0,#{},#{},undefined,rabbit_classic_queue_index_v2,
                                    {qi,{resource,<<"/">>,queue,
                                                  <<"mqtt-subscription-nuc_bench_sub_23092_1622527410qos1">>},
                                        "/home/david/scratch/rabbit/test/rabbit@nuc/mnesia/rabbit@nuc/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/queues/8Q5F1PPMH3BD1C7F6CQXGLGME",
                                        #{},0,#{},#{},#{},#{},
                                        #Fun<rabbit_variable_queue.2.85400881>,
                                        #Fun<rabbit_variable_queue.3.85400881>},
                                    {qs,"/home/david/scratch/rabbit/test/rabbit@nuc/mnesia/rabbit@nuc/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/queues/8Q5F1PPMH3BD1C7F6CQXGLGME",
                                        undefined,64,#{},0,#{},undefined,undefined},
                                    {{client_msstate,<0.712.0>,
                                                     <<189,37,176,225,122,247,125,208,243,230,50,155,167,150,
                                                       206,58>>,
                                                     #{},
                                                     {state,#Ref<0.1127660195.458620929.200967>,
                                                            "/home/david/scratch/rabbit/test/rabbit@nuc/mnesia/rabbit@nuc/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent"},
                                                     rabbit_msg_store_ets_index,
                                                     "/home/david/scratch/rabbit/test/rabbit@nuc/mnesia/rabbit@nuc/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent",
                                                     <0.715.0>,#Ref<0.1127660195.458620929.200968>,
                                                     #Ref<0.1127660195.458620929.200965>,
                                                     #Ref<0.1127660195.458620929.200969>,
                                                     #Ref<0.1127660195.458620929.200970>,
                                                     {4000,800}},
                                     {client_msstate,<0.708.0>,
                                                     <<210,61,62,233,91,44,113,162,15,92,141,3,168,202,67,57>>,
                                                     #{},
                                                     {state,#Ref<0.1127660195.458620929.200936>,
                                                            "/home/david/scratch/rabbit/test/rabbit@nuc/mnesia/rabbit@nuc/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient"},
                                                     rabbit_msg_store_ets_index,
                                                     "/home/david/scratch/rabbit/test/rabbit@nuc/mnesia/rabbit@nuc/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient",
                                                     <0.709.0>,#Ref<0.1127660195.458620929.200937>,
                                                     #Ref<0.1127660195.458620929.200934>,
                                                     #Ref<0.1127660195.458620929.200938>,
                                                     #Ref<0.1127660195.458620929.200939>,
                                                     {4000,800}}},
                                    true,0,4096,0,0,0,0,0,0,infinity,0,0,0,0,0,0,
                                    {rates,0.0,0.0,0.0,0.0,-576460124199912220},
                                    #{},#{},#{},#{},0,0,0,0,4096,default,2,#{},<<"/">>,false}},

Alternatives to the implementation in this commit:

1. Store common directory list prefix only once (e.g. put it into persistent_term) and store per queue directory names in ETS.
2. Use file:filename_all() instead of file:filename() and pass binaries to the file module functions. However this might be brittle on some platforms since these binaries are interpreted as "raw filenames". Using raw filenames requires more changes to classic queues which we want to avoid to reduce risk.

The downside of the implemenation in this commit is that the binary gets converted to a list sometimes.
This happens whenever a file is flushed or a new file gets created for example.

Following perf tests did not show any regression in performance:

java -jar target/perf-test.jar -s 10 -x 1 -y 0 -u q -f persistent -z 30
java -jar target/perf-test.jar -s 10000 -x 1 -y 0 -u q -f persistent -z 30
java -jar target/perf-test.jar -s 10 -x 100 -qp q%d -qpf 1 -qpt 100 -y 0 -f persistent -z 60 -c 1000

Furthermore rabbit_file did not show up in the CPU flame graphs either.
It showed up only once with 0.1% in a page fault flame graph.

[lhoguin]

The approach seems sensible, but I'll echo what JP said, careful about Windows and weird file names. Let's test this thoroughly.

[lukebakken]

Test process for Windows:

• Build zip file, extract locally
• Create the C:\ProgramData\RabbitMQ Sérvér directory
• Start powershell, cd to extracted RabbitMQ build
• Set the following $env:RABBITMQ_BASE='C:\ProgramData\RabbitMQ Sérvér'
• Create C:\ProgramData\RabbitMQ Sérvér\rabbitmq-env-conf.bat with this content:

  set LOG=debug
  set RABBITMQ_ALLOW_INPUT=true
  

• Start RabbitMQ, in management UI, create a v1 and v2 queue
• Start two instances of PerfTest, like this:

  mvn exec:java '-Dexec.mainClass=com.rabbitmq.perf.PerfTest' '-Dexec.args=--predeclared --queue v2'
  

• Note that publishing works fine
• Kill erlang Stop-Process -Name werl
• Restart RabbitMQ
• PerfTest reconnects, RabbitMQ starts normally