Skip to content

Commit 84edc05

Browse files
committed
Convert character offset to byte offset in parse_inline_*_annotation
Parser.parse_inline_leading_annotation and parse_inline_trailing_annotation were passing character offsets directly to the C parser, which expects byte offsets since #2863. With ASCII-only input the two coincide so the bug was hidden, but multibyte content (e.g. a Japanese comment preceding the annotation) caused the C parser to start at an invalid byte position and emit a parsing error. Apply the existing byte_range helper to convert character offsets to byte offsets, matching the pattern used by parse_type, parse_method_type, and other parse_* methods. The existing tests for parse_inline_*_annotation used ASCII-only input which would not catch this class of bug, so multibyte regression tests are added.
1 parent 4c6a3a9 commit 84edc05

2 files changed

Lines changed: 28 additions & 2 deletions

File tree

lib/rbs/parser_aux.rb

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -121,12 +121,14 @@ def self.buffer(source)
121121

122122
def self.parse_inline_leading_annotation(source, range, variables: [])
123123
buf = buffer(source)
124-
_parse_inline_leading_annotation(buf, range.begin || 0, range.end || buf.last_position, variables)
124+
byte_range = byte_range(range, buf.content)
125+
_parse_inline_leading_annotation(buf, byte_range.begin || 0, byte_range.end || buf.content.bytesize, variables)
125126
end
126127

127128
def self.parse_inline_trailing_annotation(source, range, variables: [])
128129
buf = buffer(source)
129-
_parse_inline_trailing_annotation(buf, range.begin || 0, range.end || buf.last_position, variables)
130+
byte_range = byte_range(range, buf.content)
131+
_parse_inline_trailing_annotation(buf, byte_range.begin || 0, byte_range.end || buf.content.bytesize, variables)
130132
end
131133

132134
def self.byte_range(char_range, content)

test/rbs/inline_annotation_parsing_test.rb

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -559,4 +559,28 @@ def test_error__module_self
559559
Parser.parse_inline_leading_annotation("@rbs module-self: foo", 0...)
560560
end
561561
end
562+
563+
def test_parse__trailing_assertion__multibyte_offset
564+
buffer = Buffer.new(name: Pathname("test.rb"), content: "日本語\n: String")
565+
566+
Parser.parse_inline_trailing_annotation(buffer, 4...12).tap do |annot|
567+
assert_instance_of AST::Ruby::Annotations::NodeTypeAssertion, annot
568+
assert_equal ": String", annot.location.source
569+
assert_equal ":", annot.prefix_location.source
570+
assert_equal "String", annot.type.location.source
571+
end
572+
end
573+
574+
def test_parse__leading_annotation__multibyte_offset
575+
buffer = Buffer.new(name: Pathname("test.rb"), content: "日本語のコメント\n@rbs return: String -- 戻り値の説明")
576+
577+
Parser.parse_inline_leading_annotation(buffer, 9...).tap do |annot|
578+
assert_instance_of AST::Ruby::Annotations::ReturnTypeAnnotation, annot
579+
assert_equal "@rbs return: String -- 戻り値の説明", annot.location.source
580+
assert_equal "return", annot.return_location.source
581+
assert_equal ":", annot.colon_location.source
582+
assert_equal "String", annot.return_type.location.source
583+
assert_equal "-- 戻り値の説明", annot.comment_location.source
584+
end
585+
end
562586
end

0 commit comments

Comments
 (0)