Orz日記 by Akio Morita

_ [SAD][Fortran]Fortran I/Oの怪

Fortran90から入った停留入力で制御文字"\\r"の振るまい奇妙なので、 実装ごとの挙動を調べてみました。

ここでは、"\\r","\\n","\\r\\n"などの改行記号が 入力列に含まれて際の read文からの戻り値を調べています。 サンプルとした入力列は、"abc\\rdef\\n\\nABCDEF\\r\\n123456\\neof"です。 この入力列は、標準入力(UNIT=5)とファイル経由(UNIT=10)で テストプログラムに入力され、read文で停留入力による読み取りが行われます。

g95 0.91 20080220の場合

• 標準入力とファイル経由の結果は同じ
• 単独の"\\r"は1文字として扱う
• "\\r\\n"と"\\n"は改行記号扱いで区別できない
• 複数文字を受け取る場合、改行記号とEOFを区別できない
  • 1文字づつ受け取る場合は、EOFを検知する直前が改行記号で無い場合には、size=0,iostat=-2が返って来る
  • この例だと、"e",size=1,iostat=0->"o",size=1,iostat=0->"f",size=1,iostat=0->size=0,iostat=-2->EOF検知

gfortran 4.2.4 20080220 (prerelease)の場合

• 標準入力からだと"\\r"の次が"\\n"以外の文字の場合に欠落する
  • CR-LFを仮定して LFを先読みした後に seekで戻しているが、un-seek-ableなストリームではこの実装はうまく動かない
• "\\r\\n"、"\\r"と"\\n"は改行記号扱いで区別できない
• 複数文字を受け取る場合、改行記号とEOFを区別できない
  • 1文字づつ受け取る場合は、改行記号とEOFを区別できる
  • この例だと、"e",size=1,iostat=0->"o",size=1,iostat=0->"f",size=1,iostat=0->EOF検知

gfortran 4.3.0 20080221 (prerelease)の場合

• 標準入力からだと"\\r"の次が"\\n"以外の文字の場合に欠落する
  • CR-LFを仮定して LFを先読みした後に seekで戻しているが、un-seek-ableなストリームではこの実装はうまく動かない
• "\\r\\n"、"\\r"と"\\n"は改行記号扱いで区別できない
• 複数文字を受け取る場合、改行記号とEOFを区別できない
  • 1文字づつ受け取る場合は、改行記号とEOFを区別できる
  • この例だと、"e",size=1,iostat=0->"o",size=1,iostat=0->"f",size=1,iostat=0->EOF検知
• gfortran 4.2.4 20080220 (prerelease)と同じ動作

Intel Fortran Compiler Version 8.1の場合

• 標準入力とファイル経由の結果は同じ
• 単独の"\\r"は1文字として扱う
• "\\r\\n"と"\\n"は改行記号扱いで区別できない
• 複数文字を受け取る場合、改行記号とEOFを区別できない
  • 1文字づつ受け取る場合は、EOFを検知する直前が改行記号で無い場合には、size=0,iostat=-2が返って来る
  • この例だと、"e",size=1,iostat=0->"o",size=1,iostat=0->"f",size=1,iostat=0->size=0,iostat=-2->EOF検知

まとめ

• "\\r\\n"と"\\n"が同一視されている
• 2008/02/20頃のgfortran 4.2.4/4.3.0実装は明らかにおかしい
  • 先行するCRに対応するLFを一文字読み込みして探すのは良いとして、LFで無かった際に正しくストリームを巻戻せず入力を失っている(標準入力)
• 改行記号終端でない入力でのEOF検知の挙動が実装で異なる
  • 改行記号によるレコード終端の取り扱いに差か?
  • gfortranは、EOFによる暗黙のレコード終端よりEOF検知が優先している
  • intel fortran/g95は、EOFによる暗黙のレコード終端によって一度、size=0,iostat=-2(改行〜レコード終端)を返してから、EOFを検知する

つまり、改行区切りなテキスト読み込み(itfgetbuf_)には使えるが、uni-byte stream入力には使えないと言うこと

性能比較

itfgetbuf_の実装を

• Q editing
• FGETC intrinsic
• Fortran90 停留入力
• read(2) system call

で行ったものを比較すると、read(2)が断トツに遅いのは予定通りだとしても、 停留入力がFGETCより遅いのは納得がいかないな

Intel Fortran 8.1で試すと、処理時間比は Q editing:FGETC:停留入力:read(2)で1:1.7:7:23ぐらいで、 FGETCに比べるて停留入力が4倍近く遅い... Orz

gfortranだと、FGETC:停留入力:read(2)で 1:1.6:8.9ぐらいでそれほど激しい差では無いですが停留入力の方が遅い

内容的には、I/O library側でループを組める&最適化できるので FGETCのループより早くてもよさそうなモノですが...

itfgetbuf_の実装に限れば、

1. Q編集記述子が有るならそれを使え
2. さもなくば、FGETC intrinsicを使え
3. 他に手が無ければ停留入力で書け
4. read(2)は最後の武器だ

ということですね。 実際は、binary Read[]で FGETC intrinsicが使われているので FGETCが未実装の場合、整合性を取ろうとするとread(2)で 全部実装するしかないのですが... (それでも、Write[]とFileSeek[]との完全には整合性しない)

_ [雑記]NANOHA the MOVIE 1st(2回目)

記念色紙ゲット

やはり、2度見ても良い作品だな...

流石に、観客は減ったようだが、パンフレットが一人二部までとか 書いてあったのは吹いた(転売屋が買占めでもしたのかな...)

_ [FreeBSD][Emacs]japanese/lookup壊れた

Emacs 30.1に更新したら、japanese/lookupが壊れたぽぃ

M-x lookupで単語検索すると

Looking up `foo'... (検索対象の辞書名...)
lookup-dictionary-search-cache-get: Wrong type argument: obarrayp, [nil nil nil nil nil nil nil nil nil nil ...]

で結果が戻ってこない

scratch bufferで実行したログ(2025/03/04追記)

UIにバインドされている関数lookup-find-patternを追いかけて、入力パターンを受けて実行される部分をscratch buffer上で評価して詳細なエラーログを取得してみた

(lookup-search-pattern (lookup-current-module) "test") C-j

Debugger entered--Lisp error: (wrong-type-argument obarrayp [nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil ...])
  intern("test" [nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil ...])
  lookup-dictionary-search-cache-get(#s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/readers" :options (:alias "readers") :id "ndeb:/usr/local/share/dict/cdrom/readers") :name "readers" :options nil :id "ndeb:/usr/local/share/dict/cdrom/readers/readers") #s(lookup-query :method regexp :string "test" :pattern "test"))
  lookup-dictionary-search-internal(#s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/readers" :options (:alias "readers") :id "ndeb:/usr/local/share/dict/cdrom/readers") :name "readers" :options nil :id "ndeb:/usr/local/share/dict/cdrom/readers/readers") #s(lookup-query :method regexp :string "test" :pattern "test"))
  #f(compiled-function (query) #<bytecode -0x4b416b1ee2a9b0>)(#s(lookup-query :method regexp :string "test" :pattern "test"))
  mapcar(#f(compiled-function (query) #<bytecode -0x4b416b1ee2a9b0>) (#s(lookup-query :method regexp :string "test" :pattern "test")))
  lookup-dictionary-search-multiple(#s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/readers" :options (:alias "readers") :id "ndeb:/usr/local/share/dict/cdrom/readers") :name "readers" :options nil :id "ndeb:/usr/local/share/dict/cdrom/readers/readers") (#s(lookup-query :method regexp :string "test" :pattern "test")))
  lookup-dictionary-search(#s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/readers" :options (:alias "readers") :id "ndeb:/usr/local/share/dict/cdrom/readers") :name "readers" :options nil :id "ndeb:/usr/local/share/dict/cdrom/readers/readers") #s(lookup-query :method default :string "test" :pattern "test"))
  lookup-search-query(#s(lookup-module :name "scrup" :dictionaries (#s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/readers" :options ... :id "ndeb:/usr/local/share/dict/cdrom/readers") :name "readers" :options nil :id "ndeb:/usr/local/share/dict/cdrom/readers/readers") #s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/NaturalScienceEtoJ" :options ... :id "ndeb:/usr/local/share/dict/cdrom/NaturalScienceEtoJ") :name "body" :options nil :id "ndeb:/usr/local/share/dict/cdrom/NaturalScienceEtoJ/body") #s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/NaturalScienceJtoE" :options ... :id "ndeb:/usr/local/share/dict/cdrom/NaturalScienceJtoE") :name "body" :options nil :id "ndeb:/usr/local/share/dict/cdrom/NaturalScienceJtoE/body") #s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/SuperDic2006" :options ... :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006") :name "kojien" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/kojien") #s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/SuperDic2006" :options ... :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006") :name "readers" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/readers") #s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/SuperDic2006" :options ... :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006") :name "wachu5" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/wachu5") #s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/SuperDic2006" :options ... :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006") :name "kanjigen" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/kanjigen") #s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/SuperDic2006" :options ... :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006") :name "gn06ep01" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/gn06ep01") #s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/SuperDic2006" :options ... :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006") :name "jyukugo" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/jyukugo") #s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/SuperDic2006" :options ... :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006") :name "katakana" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/katakana") #s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/SuperDic2006" :options ... :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006") :name "screen" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/screen")) :bookmarks nil :priority-alist ((#s(lookup-dictionary :agent ... :name "body" :options nil :id "ndeb:/usr/local/share/dict/cdrom/NaturalScienceEtoJ/body") . t) (#s(lookup-dictionary :agent ... :name "readers" :options nil :id "ndeb:/usr/local/share/dict/cdrom/readers/readers") . t) (#s(lookup-dictionary :agent ... :name "screen" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/screen") . t) (#s(lookup-dictionary :agent ... :name "katakana" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/katakana") . t) (#s(lookup-dictionary :agent ... :name "jyukugo" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/jyukugo") . t) (#s(lookup-dictionary :agent ... :name "gn06ep01" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/gn06ep01") . t) (#s(lookup-dictionary :agent ... :name "kanjigen" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/kanjigen") . t) (#s(lookup-dictionary :agent ... :name "wachu5" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/wachu5") . t) (#s(lookup-dictionary :agent ... :name "readers" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/readers") . t) (#s(lookup-dictionary :agent ... :name "kojien" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/kojien") . t) (#s(lookup-dictionary :agent ... :name "body" :options nil :id "ndeb:/usr/local/share/dict/cdrom/NaturalScienceJtoE/body") . t))) #s(lookup-query :method default :string "test" :pattern "test"))
  lookup-search-pattern(#s(lookup-module :name "scrup" :dictionaries (#s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/readers" :options ... :id "ndeb:/usr/local/share/dict/cdrom/readers") :name "readers" :options nil :id "ndeb:/usr/local/share/dict/cdrom/readers/readers") #s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/NaturalScienceEtoJ" :options ... :id "ndeb:/usr/local/share/dict/cdrom/NaturalScienceEtoJ") :name "body" :options nil :id "ndeb:/usr/local/share/dict/cdrom/NaturalScienceEtoJ/body") #s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/NaturalScienceJtoE" :options ... :id "ndeb:/usr/local/share/dict/cdrom/NaturalScienceJtoE") :name "body" :options nil :id "ndeb:/usr/local/share/dict/cdrom/NaturalScienceJtoE/body") #s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/SuperDic2006" :options ... :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006") :name "kojien" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/kojien") #s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/SuperDic2006" :options ... :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006") :name "readers" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/readers") #s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/SuperDic2006" :options ... :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006") :name "wachu5" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/wachu5") #s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/SuperDic2006" :options ... :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006") :name "kanjigen" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/kanjigen") #s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/SuperDic2006" :options ... :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006") :name "gn06ep01" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/gn06ep01") #s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/SuperDic2006" :options ... :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006") :name "jyukugo" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/jyukugo") #s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/SuperDic2006" :options ... :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006") :name "katakana" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/katakana") #s(lookup-dictionary :agent #s(lookup-agent :class ndeb :location "/usr/local/share/dict/cdrom/SuperDic2006" :options ... :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006") :name "screen" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/screen")) :bookmarks nil :priority-alist ((#s(lookup-dictionary :agent ... :name "body" :options nil :id "ndeb:/usr/local/share/dict/cdrom/NaturalScienceEtoJ/body") . t) (#s(lookup-dictionary :agent ... :name "readers" :options nil :id "ndeb:/usr/local/share/dict/cdrom/readers/readers") . t) (#s(lookup-dictionary :agent ... :name "screen" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/screen") . t) (#s(lookup-dictionary :agent ... :name "katakana" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/katakana") . t) (#s(lookup-dictionary :agent ... :name "jyukugo" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/jyukugo") . t) (#s(lookup-dictionary :agent ... :name "gn06ep01" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/gn06ep01") . t) (#s(lookup-dictionary :agent ... :name "kanjigen" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/kanjigen") . t) (#s(lookup-dictionary :agent ... :name "wachu5" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/wachu5") . t) (#s(lookup-dictionary :agent ... :name "readers" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/readers") . t) (#s(lookup-dictionary :agent ... :name "kojien" :options nil :id "ndeb:/usr/local/share/dict/cdrom/SuperDic2006/kojien") . t) (#s(lookup-dictionary :agent ... :name "body" :options nil :id "ndeb:/usr/local/share/dict/cdrom/NaturalScienceJtoE/body") . t))) "test")
  (progn (lookup-search-pattern (lookup-current-module) "test"))
  eval((progn (lookup-search-pattern (lookup-current-module) "test")) t)
  elisp--eval-last-sexp(t)
  #f(compiled-function () #<bytecode 0x1842a8d95a4a>)()
  handler-bind-1(#f(compiled-function () #<bytecode 0x1842a8d95a4a>) (error) eval-expression--debug)
  eval-last-sexp(t)
  eval-print-last-sexp(nil)
  funcall-interactively(eval-print-last-sexp nil)
  call-interactively(eval-print-last-sexp nil nil)
  command-execute(eval-print-last-sexp)

当然だが、Emacs 29.4上では発生しない…なんだこれ？

なんか、(lookup-current-module)の評価値が、Emacs 29.4環境とEmacs 30.1環境で違うような… .emacs回りの再現性は要確認か？