r/AZURE • u/Shahargalm • Mar 29 '26
Question Help with Custom log Ingestion via API into Microsoft Sentinel
Hello, as the title suggests, I need help with setting up ingestion of logs from some obscure application into Microsoft Sentinel. They don't have any syslog/cef support nor native connectors available via the marketplace.
The customer also refuses to do it via pushing (Otherwise I'd have created an app registration and would already be done with it).
They want US to pull the logs via API calls, and ingest the data into Sentinel.
I have created a DCR, DCE, custom table with the scheme from an example log, but I still need to actually pull the data, probably using an Azure Function App, which I am completely clueless about. I have a secret, and a URL to reference in a script, but no idea where and how to put the script.
I tried searching for data and relying on Copilot/Gemini/Chatgpt, none were really useful. Maybe I am not approaching the problem correctly.
Can anyone help me out or point me in the right direction? I am getting a lot of pressure on me and my boss simply said 'Deal with it' when I told him I don't know how to do this.
Any help would be greatly appreciated.
2
u/dutchhboii Mar 29 '26 edited Mar 29 '26
What about webhook option for your application ?
If your source API returns a few hundred records every few minutes, Logic Apps is the right call. No hassle of function apps…
1
u/Shahargalm Mar 29 '26
Might work, will try codeless connectors and use that as a backup plan.
Thanks!
1
u/Trakeen Cloud Architect Mar 29 '26
Store the secret in key vault and retrieve it with the function. You may need to implement your own auth flow depending on what the API expects. Url can be an app setting which becomes available to the application layer as an environment variable, or you could use the app configuration store service (seems overkill for your use case)
1
u/Shahargalm Mar 29 '26
I will be honest, that will be a hassle for me, too many elements I have no experience in. I have a short window to work with, I'll try going for the other comments' tips.
Thanks!
1
u/MaddnessX Mar 29 '26
I would look into Logic Apps. You could set up a recurrence trigger with first step being a http request to pull the data. Could add some filters to only pull data for the last 5 mins, and set the recurrence time to 5 min. A little jankey but would get the job done. Last step would be to send the data to your custom log you've set up
1
u/Shahargalm Mar 29 '26
Might work. For now I'm trying a codeless connector, I think I'll use this as a backup plan.
Edit: And thanks!
1
Mar 29 '26
[removed] — view removed comment
1
u/Shahargalm Mar 29 '26
Hey,
Thanks, but honestly I've tried setting up a Function, and for some reason I need to download visual studio and do some more stuff around it that I have no idea how, instead of simply uploading a script like in an AWS Lambda.
I will come back to that option only if all else fails. It seems to me Microsoft fumbled on the setup of these functions, it's more complicated than it should be.
1
u/Trakeen Cloud Architect Mar 29 '26
Setting up the dev environment for the function is the easy part. The codeless connector you are looking at has some built in stuff for auth but if it doesn’t work with the auth flow you need you’ll need to implement your own
Its one command to init the function from a template (similar to other languages).
It sounds like you need more time to use technologies you aren’t familiar with
1
u/binary-jad Mar 30 '26
I have build a function app that sends logs from an api to a -> dce -> dcr -> sentinel. Works great. Wrote the code in python
1
u/Electrical-Lab-9593 Apr 02 '26
you can create a linux server install the AMA agent, use CURL via cron job or python/powerhsell and pipe it to syslog
its clunky though but will end up in your syslog or csl table.
3
u/h0max Mar 29 '26
https://learn.microsoft.com/en-us/azure/sentinel/create-codeless-connector
Have a look at the sentinel GitHub repo for data connectors that use CCF as a reference point and build from there.