Skip to content

Ruby 3.2 support #3039

New issue
New issue
Closed
Closed
Ruby 3.2 support#3039
Assignees
nirvdrumrwstaunereregonandrykonchinhorakivo
Labels
compatibilitygood first issue
Milestone
 
24.0.0 Release (March 19, 2024)
@eregon

Description

@eregon
eregon
opened on May 5, 2023

We are working on Ruby 3.2 support for the next release (23.1).

Any help is appreciated whether it is:

  • implementing one of the Ruby 3.2 changes in TruffleRuby, and ensuring it is tested in specs or MRI tests and otherwise add specs. If adding specs please leave a comment here to mention you are working on adding these specs to avoid conflicts & duplicated work.
  • adding specs for Ruby 3.2 changes, which is best done in ruby/spec directly, see Write specs for new Ruby 3.2 features and changes ruby/spec#1016

To make it easier to find good getting-started issues, I annotated items with easy, medium and hard.
There is also pure ruby for changes which should only affect Ruby code and not require any change to Java sources or any Java knowledge.
If there is no pure ruby then some Java code is involved, but it should still be fairly straightforward if easy or medium.
hard typically requires more knowledge about TruffleRuby implementation details.
[import] means this needs the import of Ruby 3.2 sources to happen first.

If you'd like to start on some task please mention it here or edit the description directly for committers (to avoid duplicated work).

Currently RUBY_VERSION is still 3.1.x. So to work on 3.2 changes, follow this documentation.

Notes

  • Fiber scheduler changes are not implemented because it seems not worth it until Truffle supports VirtualThread on both Native Image and HotSpot.
  • Ractor is not implemented because it seems of little value. Thread already run in parallel and are far more compatible.

Full list of changes for Ruby 3.2.0

NOTE: https://rubyreferences.github.io/rubychanges/3.2.html gives more details for many features and changes.

From https://github.com/ruby/ruby/blob/master/doc/NEWS/NEWS-3.2.0.md:

This document is a list of user-visible feature changes
since the 3.1.0 release, except for bug fixes.

Note that each entry is kept to a minimum, see links for details.

Language changes

  • [parser, YARP] Anonymous rest and keyword rest arguments can now be passed as
    arguments, instead of just used in method parameters.
    [Feature #18351]

    def foo(*)
  bar(*)
end
def baz(**)
  quux(**)
end

  • [medium, java] A proc that accepts a single positional argument and keywords will
    no longer autosplat. [Bug #18633]

    proc{|a, **k| a}.call([1, 2])
# Ruby 3.1 and before
# => 1
# Ruby 3.2 and after
# => [1, 2]

  • [medium, java] (@andrykonchin) Constant assignment evaluation order for constants set on explicit
    objects has been made consistent with single attribute assignment
    evaluation order. With this code:

    foo::BAR = baz

    foo is now called before baz. Similarly, for multiple assignments
    to constants, left-to-right evaluation order is used. With this
    code:

    foo1::BAR1, foo2::BAR2 = baz1, baz2

    The following evaluation order is now used:

    1. foo1
    2. foo2
    3. baz1
    4. baz2

    [Bug #15928]

  • [parser, YARP, need to implement Find pattern in translator] "Find pattern" is no longer experimental.
    [Feature #18585]

  • @eregon Methods taking a rest parameter (like *args) and wishing to delegate keyword
    arguments through foo(*args) must now be marked with ruby2_keywords
    (if not already the case). In other words, all methods wishing to delegate
    keyword arguments through *args must now be marked with ruby2_keywords,
    with no exception. This will make it easier to transition to other ways of
    delegation once a library can require Ruby 3+. Previously, the ruby2_keywords
    flag was kept if the receiving method took *args, but this was a bug and an
    inconsistency. A good technique to find the potentially-missing ruby2_keywords
    is to run the test suite, for where it fails find the last method which must
    receive keyword arguments, use puts nil, caller, nil there, and check each
    method/block on the call chain which must delegate keywords is correctly marked
    as ruby2_keywords. [Bug #18625] [Bug #16466]

    def target(**kw)
end

# Accidentally worked without ruby2_keywords in Ruby 2.7-3.1, ruby2_keywords
# needed in 3.2+. Just like (*args, **kwargs) or (...) would be needed on
# both #foo and #bar when migrating away from ruby2_keywords.
ruby2_keywords def bar(*args)
  target(*args)
end

ruby2_keywords def foo(*args)
  bar(*args)
end

foo(k: 1)

Core classes updates

Note: We're only listing outstanding class updates.

Fiber

  • [medium, java] Introduce Fiber.[] and Fiber.[]= for inheritable fiber storage.
    Introduce Fiber#storage and Fiber#storage= (experimental) for
    getting and resetting the current storage. Introduce
    Fiber.new(storage:) for setting the storage when creating a
    fiber. [Feature #19078]

    Existing Thread and Fiber local variables can be tricky to use.
    Thread-local variables are shared between all fibers, making it
    hard to isolate, while Fiber-local variables can be hard to
    share. It is often desirable to define unit of execution
    ("execution context") such that some state is shared between all
    fibers and threads created in that context. This is what Fiber
    storage provides.

    def log(message)
  puts "#{Fiber[:request_id]}: #{message}"
end

def handle_requests
  while request = read_request
    Fiber.schedule do
      Fiber[:request_id] = SecureRandom.uuid

      request.messages.each do |message|
        Fiber.schedule do
          log("Handling #{message}") # Log includes inherited request_id.
        end
      end
    end
  end
end

    You should generally consider Fiber storage for any state which
    you want to be shared implicitly between all fibers and threads
    created in a given context, e.g. a connection pool, a request
    id, a logger level, environment variables, configuration, etc.

Fiber::Scheduler

    • [needs other fiber scheduler parts first] Introduce Fiber::Scheduler#io_select for non-blocking IO.select.
      [Feature #19060]

IO

  • [hard, java] Introduce IO#timeout= and IO#timeout which can cause
    IO::TimeoutError to be raised if a blocking operation exceeds the
    specified timeout. [Feature #18630]

    STDIN.timeout = 1
STDIN.read # => Blocking operation timed out! (IO::TimeoutError)

  • @moste00 [easy, pure ruby] Introduce IO.new(..., path:) and promote File#path to IO#path. [Feature #19036]

Class

  • [easy, java] Class#attached_object, which returns the object for which
    the receiver is the singleton class. Raises TypeError if the receiver is not a singleton class. [Feature #12084]

    class Foo; end

Foo.singleton_class.attached_object        #=> Foo
Foo.new.singleton_class.attached_object    #=> #<Foo:0x000000010491a370>
Foo.attached_object                        #=> TypeError: `Foo' is not a singleton class
nil.singleton_class.attached_object        #=> TypeError: `NilClass' is not a singleton class

Data

  • @moste00 [medium, pure ruby] New core class to represent simple immutable value object. The class is
    similar to Struct and partially shares an implementation, but has more
    lean and strict API. [Feature #16122]

    Measure = Data.define(:amount, :unit)
distance = Measure.new(100, 'km')            #=> #<data Measure amount=100, unit="km">
weight = Measure.new(amount: 50, unit: 'kg') #=> #<data Measure amount=50, unit="kg">
weight.with(amount: 40)                      #=> #<data Measure amount=40, unit="kg">
weight.amount                                #=> 50
weight.amount = 40                           #=> NoMethodError: undefined method `amount='

Encoding

  • [easy, pure ruby] Encoding#replicate has been deprecated and will be removed in 3.3. [Feature #18949]
  • [easy, java, remove GetActualEncodingNode] The dummy Encoding::UTF_16 and Encoding::UTF_32 encodings no longer
    try to dynamically guess the endian based on a byte order mark.
    Use Encoding::UTF_16BE/UTF_16LE and Encoding::UTF_32BE/UTF_32LE instead.
    This change speeds up getting the encoding of a String. [Feature #18949]
  • [medium, java] Limit maximum encoding set size by 256.
    If exceeding maximum size, EncodingError will be raised. [Feature #18949]

Enumerator

  • [easy, pure ruby] Enumerator.product has been added. Enumerator::Product is the implementation. [Feature #18685]

Exception

  • [easy, some ruby, some java] Exception#detailed_message has been added.
    The default error printer calls this method on the Exception object
    instead of #message. [Feature #18564]

Hash

  • Hash#shift now always returns nil if the hash is
    empty, instead of returning the default value or
    calling the default proc. [Bug #16908]

Integer

Kernel

  • [easy, java] Kernel#binding raises RuntimeError if called from a non-Ruby frame
    (such as a method defined in C). [Bug #18487]

MatchData

Module

  • [already done, just need to untag specs/tests] Module.used_refinements has been added. [Feature #14332]
  • [easy, java] Module#refinements has been added. [Feature #12737]
  • [medium, java] Module#const_added has been added. [Feature #17881]
  • [easy, java] Module#undefined_instance_methods has been added. [Feature #12655]

Proc

  • [easy, seems already be the case, need to untag specs] Proc#dup returns an instance of subclass. [Bug #17545]
  • [easy, java] Proc#parameters now accepts lambda keyword. [Feature #15357]

Process

  • Added RLIMIT_NPTS constant to FreeBSD platform

Regexp

  • The cache-based optimization is introduced.
    Many (but not all) Regexp matching is now in linear time, which
    will prevent regular expression denial of service (ReDoS)
    vulnerability. [Feature #19104]

  • [medium, java] Regexp.linear_time? is introduced. [Feature #19194]

  • @rwstauner [easy, pure ruby] Regexp.new now supports passing the regexp flags not only as an Integer,
    but also as a String. Unknown flags raise ArgumentError.
    Otherwise, anything other than true, false, nil or Integer will be warned.
    [Feature #18788]

  • [hard, java, would need changes in tregex backtracker] Regexp.timeout= has been added. Also, Regexp.new new supports timeout keyword.
    See [Feature #17837]

Refinement

  • [easy, java] Refinement#refined_class has been added. [Feature #12737]

Set

  • [easy, pure Ruby] Set is now available as a built-in class without the need for require "set". [Feature #16989]
    It is currently autoloaded via the Set constant or a call to Enumerable#to_set.

String

  • [easy, java] String#byteindex and String#byterindex have been added. [Feature #13110]
  • [medium, depends on jcodings] Update Unicode to Version 15.0.0 and Emoji Version 15.0. [Feature #18639]
    (also applies to Regexp)
  • [medium, java] String#bytesplice has been added. [Feature #18598]
  • [easy, pure ruby] String#dedup has been added as an alias to String#-@. [Feature #18595]

Struct

  • @rwstauner [easy, pure ruby] A Struct class can also be initialized with keyword arguments
    without keyword_init: true on Struct.new [Feature #16806]

    Post = Struct.new(:id, :name)
Post.new(1, "hello") #=> #<struct Post id=1, name="hello">
# From Ruby 3.2, the following code also works without keyword_init: true.
Post.new(id: 1, name: "hello") #=> #<struct Post id=1, name="hello">

Thread

  • [medium, java] Thread.each_caller_location is added. [Feature #16663]

Thread::Queue

  • [medium, java] Thread::Queue#pop(timeout: sec) is added. [Feature #18774]

Thread::SizedQueue

  • [medium, java] Thread::SizedQueue#pop(timeout: sec) is added. [Feature #18774]
  • [medium, java] Thread::SizedQueue#push(timeout: sec) is added. [Feature #18944]

Time

  • @rwstauner [easy, pure ruby] Time#deconstruct_keys is added, allowing to use Time instances
    in pattern-matching expressions [Feature #19071]

  • [medium] Time.new now can parse a string like generated by Time#inspect
    and return a Time instance based on the given argument.
    [Feature #18033]

SyntaxError

TracePoint

  • TracePoint#binding now returns nil for c_call/c_return TracePoints.
    [Bug #18487]
  • [medium, java, target_thread not yet implemented] TracePoint#enable target_thread keyword argument now defaults to the
    current thread if a block is given and target and target_line keyword
    arguments are not passed. [Bug #16889]

UnboundMethod

  • @rwstauner [easy, java] UnboundMethod#== returns true if the actual method is same. For example,
    String.instance_method(:object_id) == Array.instance_method(:object_id)
    returns true. [Feature #18798]

  • @rwstauner [easy, ruby+java] UnboundMethod#inspect does not show the receiver of instance_method.
    For example String.instance_method(:object_id).inspect returns
    "#<UnboundMethod: Kernel#object_id()>"
    (was "#<UnboundMethod: String(Kernel)#object_id()>").

Stdlib updates

CGI

  • [import] CGI.escapeURIComponent and CGI.unescapeURIComponent are added.
    [Feature #18822]

Coverage

  • [hard, java] Coverage.setup now accepts eval: true. By this, eval and related methods are
    able to generate code coverage. [Feature #19008]

  • [easy, pure ruby] Coverage.supported?(mode) enables detection of what coverage modes are
    supported. [Feature #19026]

Date

  • [easy, pure ruby] Added Date#deconstruct_keys and DateTime#deconstruct_keys same as [Feature #19071]

FileUtils

  • [import] Add FileUtils.ln_sr method and relative: option to FileUtils.ln_s.
    [Feature #18925]

IRB

  • [import] debug.gem integration commands have been added: debug, break, catch,
    next, delete, step, continue, finish, backtrace, info
    * They work even if you don't have gem "debug" in your Gemfile.
    * See also: What's new in Ruby 3.2's IRB?
  • [import] More Pry-like commands and features have been added.
    * edit and show_cmds (like Pry's help) are added.
    * ls takes -g or -G option to filter out outputs.
    * show_source is aliased from $ and accepts unquoted inputs.
    * whereami is aliased from @.

Net::Protocol

Pathname

Socket

  • Added the following constants for supported platforms.
    * SO_INCOMING_CPU
    * SO_INCOMING_NAPI_ID
    * SO_RTABLE
    * SO_SETFIB
    * SO_USER_COOKIE
    * TCP_KEEPALIVE
    * TCP_CONNECTION_INFO

SyntaxSuggest

  • [won't do until it uses Prism, it seems https://github.com/ruby/syntax_suggest master uses Prism now] The feature of syntax_suggest formerly dead_end is integrated in Ruby.
    [Feature #18159]

UNIXSocket

  • Add support for UNIXSocket on Windows. Emulate anonymous sockets. Add
    support for File.socket? and File::Stat#socket? where possible.
    [Feature #19135]

  • [import] The following default gems are updated.

    • RubyGems 3.4.1
    • abbrev 0.1.1
    • benchmark 0.2.1
    • bigdecimal 3.1.3
    • bundler 2.4.1
    • cgi 0.3.6
    • csv 3.2.6
    • date 3.3.3
    • delegate 0.3.0
    • did_you_mean 1.6.3
    • digest 3.1.1
    • drb 2.1.1
    • english 0.7.2
    • erb 4.0.2
    • error_highlight 0.5.1
    • etc 1.4.2
    • fcntl 1.0.2
    • fiddle 1.1.1
    • fileutils 1.7.0
    • forwardable 1.3.3
    • getoptlong 0.2.0
    • io-console 0.6.0
    • io-nonblock 0.2.0
    • io-wait 0.3.0
    • ipaddr 1.2.5
    • irb 1.6.2
    • json 2.6.3
    • logger 1.5.3
    • mutex_m 0.1.2
    • net-http 0.3.2
    • net-protocol 0.2.1
    • nkf 0.1.2
    • open-uri 0.3.0
    • open3 0.1.2
    • openssl 3.1.0
    • optparse 0.3.1
    • ostruct 0.5.5
    • pathname 0.2.1
    • pp 0.4.0
    • pstore 0.1.2
    • psych 5.0.1
    • racc 1.6.2
    • rdoc 6.5.0
    • readline-ext 0.1.5
    • reline 0.3.2
    • resolv 0.2.2
    • resolv-replace 0.1.1
    • securerandom 0.2.2
    • set 1.0.3
    • stringio 3.0.4
    • strscan 3.0.5
    • syntax_suggest 1.0.2
    • syslog 0.1.1
    • tempfile 0.1.3
    • time 0.2.1
    • timeout 0.3.1
    • tmpdir 0.1.3
    • tsort 0.1.1
    • un 0.2.1
    • uri 0.12.0
    • weakref 0.1.2
    • win32ole 1.8.9
    • yaml 0.2.1
    • zlib 3.0.0

  • [import] The following bundled gems are updated.

    • minitest 5.16.3
    • power_assert 2.0.3
    • test-unit 3.5.7
    • net-ftp 0.2.0
    • net-imap 0.3.4
    • net-pop 0.1.2
    • net-smtp 0.3.3
    • rbs 2.8.2
    • typeprof 0.21.3
    • debug 1.7.1

See GitHub releases like GitHub Releases of Logger or changelog for details of the default gems or bundled gems.

Compatibility issues

  • [easy, pure ruby] String#to_c currently treat a sequence of underscores as an end of Complex
    string. [Bug #19087]

  • [easy, pure ruby] Now ENV.clone raises TypeError as well as ENV.dup [Bug #17767]

Removed constants

The following deprecated constants are removed.

  • [easy] Fixnum and Bignum [Feature #12005]
  • [easy, pure ruby] Random::DEFAULT [Feature #17351]
  • [easy, pure ruby] Struct::Group
  • [easy, pure ruby] Struct::Passwd

Removed methods

The following deprecated methods are removed.

Error printer

  • Ruby no longer escapes control characters and backslashes in an
    error message. [Feature #18367]

Constant lookup when defining a class/module

  • [medium, java] When defining a class/module directly under the Object class by class/module
    statement, if there is already a class/module defined by Module#include
    with the same name, the statement was handled as "open class" in Ruby 3.1 or before.
    Since Ruby 3.2, a new class is defined instead. [Feature #18832]

Stdlib compatibility issues

  • Psych no longer bundles libyaml sources.
    And also Fiddle no longer bundles libffi sources.
    Users need to install the libyaml/libffi library themselves via the package
    manager like apt, yum, brew, etc.

    Psych and fiddle supported the static build with specific version of libyaml
    and libffi sources. You can build psych with libyaml-0.2.5 like this.

    $ ./configure --with-libyaml-source-dir=/path/to/libyaml-0.2.5

    And you can build fiddle with libffi-3.4.4 like this.

    $ ./configure --with-libffi-source-dir=/path/to/libffi-3.4.4

    [Feature #18571]

  • [import] Check cookie name/path/domain characters in CGI::Cookie. [CVE-2021-33621]

  • [import] URI.parse return empty string in host instead of nil. [sec-156615]

C API updates

Added C APIs

  • [easy, C, can just ignore the argument] VALUE rb_hash_new_capa(long capa) was added to created hashes with the desired capacity.
  • [won't do, CRuby/GIL specific] rb_internal_thread_add_event_hook and rb_internal_thread_add_event_hook were added to instrument threads scheduling.
    The following events are available:
    • RUBY_INTERNAL_THREAD_EVENT_STARTED
    • RUBY_INTERNAL_THREAD_EVENT_READY
    • RUBY_INTERNAL_THREAD_EVENT_RESUMED
    • RUBY_INTERNAL_THREAD_EVENT_SUSPENDED
    • RUBY_INTERNAL_THREAD_EVENT_EXITED
  • [medium, C] rb_debug_inspector_current_depth and rb_debug_inspector_frame_depth are added for debuggers.

Removed C APIs

The following deprecated APIs are removed.

  • [easy] rb_cData variable.
  • [import] "taintedness" and "trustedness" functions. [Feature #16131]

Implementation improvements

  • [probably already done, need to check specs/tests for it] Fixed several race conditions in Kernel#autoload. [Bug #18782]

Metadata

Metadata

Assignees

Labels

compatibilitygood first issue

Type

No type

Projects

TruffleRuby Roadmap

Status

Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions