The aim of present work is to control the coupled tank system with Multi Input Multi Output (MIMO) structure by using the Sliding Mode Control (SMC) method. The most important disadvantage of the conventional SMC is chattering problem. The purpose of the presented study is to decrease the chattering effect. Two different controllers are proposed: Firstly, conventional SMC is designed. In the second step; to decrease the chattering problem in the conventional SMC, the same system is controlled by the second order SMC (SOSMC). The designed controllers have been applied to the coupled tank system, which has an important place in the industry. At the end of the study, by comparing the experimental results, it has been observed that the chattering in the control signal is significantly decreased.