Joe and Michael discuss the impact of fragmentation on databases and the importance of append-only methods to prevent fragmentation. They highlight the benefits of pre-allocating files for sequential reads and writes to maintain non-fragmented disk space.