OK, to answer the question, we need to dig a little deeper. In reality, at the operating system level, pressing any key doesn't do anything like "deleting the next character" or insert a character into the line you are typing. In reality, at the level of the (perhaps virtual) terminal, pressing a key emits a certain sequence of characters. For example, pressing the key labeled "h" will emit the character "h", a single byte with numeric value 0x68. Some keys emit short sequences (like characters or digits), some even emit the zero byte (typically control + spacebar). But because there are more keys than can be reasonably mapped to ASCII, some keys emit complex escape sequences, for example F8 on my machine emits "esc [ 1 9 ~".
What happens next is that there is some program running, which reads the emitted character, and does something with it. For example, that program might be a shell: if they get the character "h" in their input (from the (virtual) terminal, meaning from the keyboard) will simply add the character "h" to the current command line. But that doesn't always have to be true. For example the vi editor, when it is in certain modes, will instead of adding an "h" to the document you are editing move the cursor left when it gets an "h" from the keyboard.
And at that level is the problem with the delete keys. Most modern keyboard have two "delete" keys. One is in the main alphabetic section, two keys above the return and to the right of "-" and "=". That one is sometimes also known as the backspace key. It may be labeled with the word "delete", or it may have an arrow going left on it. Traditionally, it emits either of two single-character codes: either 0x08 (which is the same as control-H, and technically called BS or backspace), or 0x7F (which is the same as control-"?", really control and question mark, which on most keyboards is done as control-shift-/, and technically called DEL or delete). Many programs such as shells are programmed and configured to react to these character codes by deleting the character to the left of the cursor. But not always: the emacs editor, for example, will delete left when it gets 0x7F = DEL = delete, but will bring up a help menu when it gets 0x08 = Control-H = BS = Backspace. This used to be a continuous source of frustration for VAX users, when they connected their VT100/VT220 to a Unix machine: deleting characters worked fine at the shell, and the moment you start the editor, you can't delete characters any longer.
The second delete key is typically in the bottom left of the block of 6 keys above the four cursor movement keys. It is often configured to emit the characters "esc [ 3 ~". Not all keyboards have that key; many laptops don't bother with it, and on traditional VT2xx layouts, it's called "remove", but is in the same position and emits the same characters. Now the question is: what will programs such as shells and editors do with it? And here lies the problem that is fixed by "bindkey": By default, some shells don't know what to do with this sequence, so they just take it literally and insert it into the line; using bindkey teaches the shell to do something sensible if it sees that sequence. In contrast, most modern editors come pre-configured so they react to that character by deleting a character to the right of the cursor (or under the cursor, not to the left).
In practice today, hardly anyone actually uses physical terminals (like VT220 models, which I still have at home); instead nearly all keyboard input is done using a virtual terminal, either in a windowing system (I'm typing this using iTerm on a Mac, on X various flavors of xterm are popular, and on Windows putty has a high market share). And even the physical keyboard and screen that are connected to an industry-standard PC are actually exposed through a virtual terminal provided by a console application, which on FreeBSD is called either vt or sc (you can read their man pages). All these virtual terminals can be configured (with variously high pain thresholds) to emit particular character codes for particular key sequences. (Historical footnote: Some good physical terminals can also be configured; the VT220 I have at home is actually not made by Digital Equipment, but is a clone model Falco 5220, which is highly configurable.)
So in summary: If you don't like what your "delete" keys do, then either configure your virtual terminal to emit the character sequence you want, or you can configure your applications (like shells and editors) to do what you want with the character sequences given to them. While "bindkey" can be part of that solution, it is only a small part.
