Sudden infant death syndrome (SIDS) is the major cause of death for infants aged one week to twelve months. The SIDS rate has declined owing to the awareness of caregivers and parents, but the rate is still high even in developed countries because of the difficulty in rescuing the infant immediately. Respiration, which can reflect various physiological conditions, is a basic but vital function for infants. Therefore, this study presents a respiration monitoring system with a video camera positioned in front of an infant to non-invasively detect the infant's respiratory frequency. The proposed system can continuously monitor the infant to detect unusual occurrences in the infant's respiration, to alert caregivers to attend to the infant immediately and reduce potential injuries from SIDS and other respiratory-related disease. The proposed system contains four major stages, including motion detection, candidate point extraction, respiration point selection, and respiratory frequency calculation. During motion detection the system captures images from video and decides whether to conduct the following stages. If no obvious motion is detected in the input frames, then SIDS may have occurred in the infant, and the system extracts candidate points by some spatial characteristics. Based on these points, the system then selects respiration points using a fuzzy integral technique with four temporal characteristics, including entropy, period, skewness, and kurtosis. Finally, the infant's respiratory frequency is calculated. Experimental data are obtained from ten infants, in 48 sequences with a total length of 150 minutes. The experimental results show that the proposed system is robust and efficient.