By default, Monitor will only ignore certain alerts during maintenance windows. You can see the list here: List of alerts suspended during maintenance windows.

You can obtain a list of the available alerts by running the following query in SSMS:

SELECT * FROM [RedGateMonitor].alert.Alert_Type

where [RedGateMonitor] is the name of your SQL Monitor database.

Then simply add the alert you wish to exclude to the exclusion list using:

UPDATE settings.KeyValuePairs
SET KeyValue = KeyValue + ',"15"'
WHERE KeyName = 'MaintenanceWindow-AlertsToExclude'