Preserve public/secret distinction in _serialize_signature_packet_target