This heavily depends on the concrete type of patch panel.
In general each additional connector has an influence on the signal quality (line attenuation, transfer resistance...). Depending on the type of panel it might have either just the sockets installed where you have to add your cables yourself or it comes with already pre-mounted cables. If you want to use GBE (Gigabit Ethernet) you need to use Category 5e cables or better.
The type of shielding also affects signal quality as well as the cable length of course.
So without knowing the concrete panel type it's hard to say whether it's safe to use GBE connections on it.
Caution: If you use short cabling and you're able to establish GBE connections via the patch panel this does not mean that the connection will be stable. So for home use and non-critical installations it might be sufficient. But for critical infrastructure I would rather recommend replacing the panels with properly-rated ones.