The objective of this paper is to investigate the failure process of brittle rock in particle scale under compression with different confining pressures. Three-dimensional discrete element code PFC3D is adopted to perform the numerical simulations where intact rock is represented by an assembly of rigid spheres bonded at their contacts. The numerical model is firstly calibrated by comparing the simulation results to the laboratory data of Beishan Granite. Good agreement can be found between the numerical and experimental results in terms of uniaxial compression strength, Young’s modulus and Poisson’s ratio. Responses at particle scale are further investigated in order to gain insights of the micro mechanisms of the failure process of brittle rock. Increment of micro cracks is examined under different confining pressures. The development of micro cracks to a major fracture is investigated by exploring the location of cracks occurring at different strain intervals. Orientation distribution of micro cracks is linked to the orientation distribution of stress acting on parallel bonds. Moreover, the effects of confining pressure are examined by comparing results from uniaxial compression test to those of triaxial compression tests. With the increase of confining pressure, the micro crack failure mode changes from tensile to shear.