Asthma is a chronic respiratory disease, in which symptoms appear or intensify suddenly, even when patients are being monitored by doctors. Continuous measurement is important to monitor a patient’s breathing without missing asthma attacks. In this study, we propose a method of continuous breathing monitoring in daily life using a wearable device. There are several studies using microphones to continuously monitor breathing during activities, which show various possibilities of extracting qualitative characteristics related to asthma. Other studies on breathing measurement using accelerometers or belts have achieved breathing detection and measurement without ambient acoustic noises. Taking advantage of the breathing sound and chest movement signals, they are simultaneously acquired using a chest-mounted device, which consists of a microphone, a photoreflector, and a flexible cover. Various acoustic noises and body movements are present in the environment. Thus, acquiring these two different signals in a complementary manner makes it easier to detect breathing in daily life. For monitoring asthma, we focused on detecting the breathing phases. Most of the asthma symptoms appear during the exhalation phase. Thus, phase detection plays an important role as an asthma symptom identifier. We developed a new algorithm for breathing phase measurement using both acquired signals. The algorithm is based on the periodicity of the chest movement signal. Breathing sounds are analyzed considering their frequency characteristics. In this paper, the basic performance of the proposed device in an experimental condition which is quiet and without participant movements is examined. The results of performance evaluation confirm that the left medial side of the second intercostal space is appropriate for placing the device and studying the correlation between breathing sound amplitude and tidal volume, which implies a potential to acquire tidal volumes. The phase measurement experiment shows that chest movement can be used for estimating the breathing period. The portable system developed can measure breathing in external conditions and tracking the wearer’s location. Making the system portable expands the measurable situations and facilitates an acquisition of time and location information, which is useful in identifying the causes of asthma attacks.