In the process of automatically processing fish body， fish body orientation and sorting is the basis of pretreating fish body. In this article， a machine vision-based fish body orientation and sorting device was designed to solve the problems of noisy working environment and poor working stability of fish orientation and sorting devices existed. The convolutional neural network Resnet-18 was used to identify the head and tail orientation of the fish， and the image processing method was used to extract the outline of the fish body and perform grid division， then the ventral and dorsal orientation of the fish was judged by comparing the size of the grey scale value. When the tail of the fish was detected to be facing forward， the cylinder pushed it out and rejected it to achieve the head and tail orientation of the fish. When the head of the fish was facing forward， the ventral and dorsal orientation of the fish was judged by the image processing， and then the cylinder pushed it to the edge of the limit plate to bring its belly against the edge， and it fell into the W-shaped chute device under the action of inertia to complete the ventral and dorsal orientation of the fish. The results showed that the success rate of fish orientation was 93.3%. The average recognition time of the algorithm was 0.038 s， and the average orientation time of each fish was 3.725 s. Theoretically， the efficiency of fish orientation can reach 15-16 fish/min， which meets the production requirements of fish orientation. It will provide useful references for the application of fish orientation and machine vision in fishery production.