The contents of an Emacs frame’s titlebar is controlled by the variable
frame-title-format, which has the same structure as the variable
mode-line-format. (Use C-h v or M-x
describe-variable to get information about one or both of these
variables.)
By default, the titlebar for a frame does contain the name of the buffer
currently being visited, except if there is a single frame. In such a
case, the titlebar contains Emacs invocation name and the name of the
machine at which Emacs was invoked. This is done by setting
frame-title-format to the default value of
(multiple-frames "%b" ("" "%b - GNU Emacs at " system-name))
To modify the behavior such that frame titlebars contain the buffer’s name regardless of the number of existing frames, include the following in your init file (see How do I set up an init file properly?):
(setq frame-title-format "%b")