Minimally Invasive Surgery (MIS) has great progress from the end of the 20th century. MIS needs not only skilled surgeons, but also accurate equipment to assist surgeons. Endoscope, as the eyes of an MIS surgeon, is to provide a 3D environment. 2D to 3D conversion adopts the video sequence captured from a single-channel 2D endoscope and calculates the depth information for the organs and surgical instruments therein to provide 3D image on stereoscopic display. This 2D to 3D conversion will assist surgeons for providing high surgical accuracy and reduce the surgical time needed.