Some files are neither directories nor regular files: they are unusual files like symbolic links, device special files, named pipes, and sockets. Currently, diff treats symbolic links as if they were the pointed-to files, except that a recursive diff reports an error if it detects infinite loops of symbolic links (e.g., symbolic links to ..). diff treats other special files like regular files if they are specified at the top level, but simply reports their presence when comparing directories. This means that patch cannot represent changes to such files. For example, if you change which file a symbolic link points to, diff outputs the difference between the two files, instead of the change to the symbolic link.
diff should optionally report changes to special files specially, and patch should be extended to understand these extensions.