The present discloses a door security system utilizing pattern recognition technology of portable device. The system comprises a portable device and a control mainframe. The portable device comprises a first wireless transmission module and can be used for telephone communication by a user. Besides, the portable device further provides a pattern decoding interface for the user to input decoding pattern. The control mainframe comprises a microcomputer, a door lock controlling interface and a first wireless transmission module. The first wireless transmission module can communicate with the second wireless transmission module at a short distance. Wherein, the decoding pattern is converted to a decoding signal and transmitted to the control mainframe by the portable device through the first wireless transmission module. The decoding signal is received by the control mainframe through the second wireless transmission module. Then, the microcomputer transmits the decoding signal to the door lock controlling interface on detecting the decoding signal. The door lock controlling interface determines whether or not to open the door lock according to the decoding signal.