Abstract—Most prior research in single speech separation has utilized time-frequency domain representation of the mixed signal. However, this approach has several drawbacks, such as separation of magnitude and phase and suboptimal speech separation quality. Our work proposes an improved model based on Conv-Tasnet architecture, which uses a full convolution time–domain auto-encoder for speech separation. The proposed model addresses previous works’ drawbacks by improving the auto-encoder’s structure to enhance the speech separation process and quality. The significant contribution of the proposed work is to preserve the magnitude and phase integrity through time-domain representation; for better gradient flow and stability, we have employed residual and skip connections. Furthermore, we have experimented with the data set Texas Instruments/Massachusetts Institute of Technology (TIMIT); Gaussian noise is added to the data set, with Signal-to-Noise Ratio (SNR) levels from −5db to 5db randomly, and the results demonstrate the improved Perceptual Evaluation of Speech Quality (PESQ), Scale-Invariant Signal-to-Noise Ratio Improvement (SI-SNRI), Scale-Invariant Signal-to- Distortion Ratio Improvement (SDRI), Signal to Distortion Ratio (STOI). The proposed model, Deep Speak net, performs state-of-the-art approaches, increases computational efficiency, and is suitable for real-time applications.