All an impedance mismatch will result in is a lower attainable maximum level. An exact impedance match provides the most efficient power transfer from the output stage to the transducer and will result in higher listening levels. If that isn't important, and with today's devices putting out too high levels as is, your ears will probably thank you for it. Keep in mind that relatively small mismatches will not result in any noticeable changes.