At present, with the rapid development of AI, people can efficiently deploy excellent neural network algorithms on FPGA accelerators by exploring the hardware design space. However, due to the large amount of parameters and complex operation, it is difficult to match the algorithm with the hardware, and the acceleration efficiency is low. In order to better match the algorithm and hardware, this paper proposed a multi-level co-exploration method, adopted SPOS search strategy, aimed at accuracy and latency, to obtain the optimal neural network architecture, quantization method and hardware design combination. It applied the method to hourglass network which had high accuracy in pose estimation. While obtained the accuracy before and after quantization of candidate sub networks, it used traversal method to search hardware design parameters and obtain the estimated latency, and then got the optimal combination with the highest score according to the target function. In order to ensure the effectiveness of the obtained data, it retrained the sub network, then quantified and inferenced again to obtain the accuracy. It simulated the obtained hardware design parameters to get the testing latency, using the accelerator template designed based on Spinal HDL. On average, co-exploration method reduced the parameters by 83.3% and with only 0.69 accuracy loss compared with the original structure; reduced the parameters by 33.2%, with only 0.46 accuracy loss, reduced the total testing latency of network inference by 22.1% and reduced the testing latency in hourglass block by 67.8% compared with the traditional acceleration method. Overall, the proposed co-exploration method has a certain effect on the optimization of hourglass network, and it has more advantages than the traditional acceleration method.