Skip to content

Conversation

@tenderlove
Copy link
Member

@tenderlove tenderlove commented Jan 23, 2024

Freeze strings and row records so that we can more easily share results among Ractors

Silly example, but here's an example of using database results plus Ractors to make a parallel producer-consumer pattern:

require "sqlite3"
require "securerandom"

def fib n
  if n < 2
    n
  else
    fib(n-2) + fib(n-1)
  end
end

N = 10

# Make a Ractor safe queue that supports N workers
queue = Ractor.new do
  while (work = Ractor.receive)
    Ractor.yield work
  end
  # Shutdown the workers
  N.times { Ractor.yield nil }
end

# Spin up workers
workers = N.times.map do
  Ractor.new(queue) do |q|
    while (work = q.take)
      name, i = work
      p "Ractor: #{Ractor.current} #{name}: #{fib(i)}"
    end
  end
end

db = SQLite3::Database.new ":memory:"
db.execute 'CREATE TABLE "fibs" (name string, fib int)'
stmt = db.prepare "INSERT INTO fibs (name, fib) VALUES (?, ?)"
5000.times { stmt.execute SecureRandom.hex, rand(30) }
stmt.close

stmt = db.prepare 'SELECT name, fib FROM fibs'

# Send work from the database to the queue
stmt.each { |row| queue.send row }

stmt.close

# Tell the queue we've sent all the work
queue.send nil

# Wait for workers to finish
workers.map(&:take)

Freeze strings and row records so that we can more easily share results
among Ractors
@tenderlove tenderlove mentioned this pull request Jan 24, 2024
19 tasks
@tenderlove
Copy link
Member Author

This change breaks one test in Rails, so I changed Rails: rails/rails#50859

@tenderlove tenderlove merged commit 44ab2aa into main Jan 24, 2024
@tenderlove tenderlove deleted the freeze-result branch January 24, 2024 02:35
if (internal_encoding) {
val = rb_str_export_to_enc(val, internal_encoding);
}
rb_obj_freeze(val);
Copy link
Member

@flavorjones flavorjones Apr 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, this seems to break Rails's schema cache tests (which use Marshal):

bin/test test/cases/connection_adapters/schema_cache_test.rb -n/test_gzip_dumps_identical/

Without this line, the test passes. With it, the test fails (in isolation).

I'm looking into it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Context on that test: rails/rails#49166

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See rails/rails#51581 for the exciting conclusion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants