Implementing Custom OAuth with NextAuth in a Next.js Frontend and Separate Backend Setup

Frontend
1

Need help with NextAuth and custom OAuth in a Next.js frontend setup

I’m working on a project where I’m using Next.js for the frontend only. The backend is a custom Java service, and we’re using a third-party OAuth authorization server. I want to implement NextAuth for authentication, but I’m struggling with the OAuth flow.

Here’s what I’m trying to figure out:

  1. How do I build the URL for the authorization server?
  2. After getting the authorization code, how should I send it to my Java backend?
  3. Once I get the JWT from the backend, how do I store it in NextAuth’s session?

I’ve tried a couple of approaches:

  • Sending the user directly to the auth server from the frontend
  • Routing through the backend first

But I’m unsure about the best way to handle redirects and data flow, especially with keeping the frontend and backend separate.

I’m using Next.js 15.1.6, NextAuth 4.24.11, and React 19.0.0.

Any tips on implementing this securely without mixing frontend and backend responsibilities would be super helpful. Thanks!

2

hey alex, that setup seems challenging. have u thought about a custom nextauth provider? storing the jwt might be easier using specific callbacks in nextauth config. im curious, what motivated u to keep the backend and frontend so separate? any insights?

3

alex, custom oauth flows can be tricky. for the auth url, try building it in your nextauth config using the authorizationParams. after getting the code, u could use a custom signin callback to send it to ur java backend. storing the JWT? look into the jwt callback in nextauth. hope this helps!

4

Implementing custom OAuth with NextAuth in a setup like yours can indeed be challenging. For building the authorization URL, consider using NextAuth’s built-in OAuth provider with custom parameters — this allows you to specify your third-party OAuth server details more clearly.

Regarding sending the authorization code to your Java backend, you could utilize NextAuth’s signIn callback to make an API call that passes the code securely. For storing the JWT, relying on the session callback within NextAuth is advisable, as it permits customization of the session object to include the JWT received from your backend.

Additionally, ensure you implement proper refresh token logic with the JWT callback to maintain session integrity over time while keeping a clear separation between the frontend and backend responsibilities.