Fractional calculus is the generalization of integer-order calculus, where the order is defined in $\mathbb{R}_+$. Paradoxically, for non-integer order's, these operators are non-local. As a result they are best suited to model non-local processes, such as turbulence. However, one can ask, \textit{whether all non-local processes can be modeled using fractional operators?} The answer, is clearly, no. Thus, one seeks further generalization of fractional operators. In order to generalize the fractional operator, Sonine recognized a key property in the Abel integral formula, that the convolution of kernel's of derivative and integral type operators is unity, thereby proposed a condition for any kernels, which is now refereed as Sonine condition. Following Sonine condition, Kochubei introduced a set of conditions for the kernels. As a result, any differential and integral operator defined with these kernel shall satisfy the fundamental theorems of calculus, thereby the birth of "general fractional calculus". However, the conditions imposed on these kernels by Kochubei were in Laplace domain, which is inconvenient to use for practical purposes. Thus, Luchko generalized the Sonine condition and termed it as modified Sonine condition. Recently, the idea's of Luchko were extended on finite interval for arbitrary orders. Thus, the mathematical theory of general fractional calculus is now complete . Indeed, differential and integral operator defined by kernels satisfying the modified Sonine condition, shall satisfy the fundamental theorems of calculus. On the numerical side, the computation of convolution-type operators, especially with singular kernel poses a challenge to construct an efficient and accurate (higher-order) scheme. Recently, we introduced the "Jacobi convolution series" (JCS), as basis functions. As a result, we obtain a higher-order Petrov-Galerkin scheme with a diagonal stiffness matrix. Furthermore, we extended our numerical approach within the paradigm of physics-informed neural networks. In this talk, we shall first review the theory of general fractional calculus, followed by our numerical work on spectral methods and general fractional physics-informed neural networks (GFPINNs).