Stephen Colbert went on "Jimmy Kimmel Live!” Tuesday night and revealed if he ever thinks about returning to late night television.