REBCO (Rare Earth Barium Copper Oxide) coated conductors (CCs) have emerged for future high field magnets in fields and temperatures inaccesible for Nb based superconductors. However, their exceptionally high current densities pose challenges for characterization at low temperatures. This paper presents the design and implementation of a simple torque magnetometer especially suitable for characterizing REBCO CC. It details the construction and underlying physics, with particular emphasis on its capability to assess angular critical currents Ic in high magnetic fields and low temperatures. The study includes characterizations of multiple REBCO samples from different manufacturers, performed under magnetic fields up to 45 T, demonstrating the exceptional capabilities of REBCO CCs in extreme fields. The results reveal significant lengthwise Ic variations, especially in tapes cut from the edges of 12 mm-wide production tapes compared to those cut from the center. These variations are most pronounced when the field is in the vicinity of the ab-plane. Importantly, flux jumps are observed in samples with thick REBCO layers and thin stabilizers, underscoring potential thermal instabilities. These findings provide valuable insights into REBCO tape performance under extreme magnetic fields, highlighting their relevance for high-field magnet and nuclear fusion applications.