Big-endian systems store the most significant byte of a word in the smallest address and the least significant byte is stored in the largest address. Little-endian systems, in contrast, store the least significant byte in the smallest address.
For example, there is a number in hexadecimal: 0x01234567, with the most significant byte 0x01 and least significant byte 0x67. It is stored within the address range 0x100 through 0x103.
Different processors may follow different conventions. For example the Intel x86 and x86-64 series of processors use the little-endian format while the Motorola 6800 and 68k series of processors use the big-endian format. And newer versions of ARM processors support bi-endian. The most important thing is the consistency. One should keep the used convention in mind when (1) binary data are communicated over a network between different machines; (2) looking at the byte sequences representing integer data, say inspecting machine-level code generated by a disassembler; (3) programs are written that circumvent the normal type system, say using a data type cast in C.