You can work them out yourself on graph paper, but I found this little Windows program to be very helpful both for generating the data bytes for custom LCD characters, and for giving you a preview of what the character will look like at its real size:
LCD Character Generator
Jim Remington's LCD bar graph program is a good example of how to store custom character arrays in AVR program memory (optional, but probably worth it if you're not generating them on the fly), load them to the LCD, and display them on the screen.
P.S. The program is a little odd in that it's a standalone executable that's actually an encrypted self-extracting zip file (so your virus scanning program can't take a look at it until you actually up and run it!). I don't think this was done maliciously, and my up-to-date McAfee doesn't take issue with the program files that get temporarily unpacked to "C:\Documents and Settings\Username\Local Settings\Temp\ir_ext_temp_n", or the running application itself.