Yes — oversizing the inverter relative to your panels can actually hurt performance, which catches people out because the usual advice is about oversizing the panels. If the inverter is too large for the array, the string voltage from your panels may not reliably climb above the threshold the inverter needs to start generating, especially in low light. In that case a smaller inverter, matched to the array, starts earlier in the day and performs better.
The healthy relationship is usually the other way round: a modest inverter with a somewhat larger panel array, so the panels keep the inverter fed across the day and through winter. The thing that actually damages an inverter isn't too much power — it's voltage exceeding the inverter's rated maximum per string. So the design job is matching your panel string voltage and current to the inverter's MPPT range, not just picking the biggest inverter available.
If an installer recommends a much larger inverter "to be safe," ask them to show the string voltage at low-light conditions and confirm it stays in the inverter's working range. Getting this wrong in the oversized direction means lazy mornings and sluggish dull-day output. The point: a bigger inverter is not automatically better, and the right size is a calculation, not a guess.