Copilot Studio bots can use generative AI to go trough pages and data and offer chatbot user interface for those. With Microsoft products creation of a chatbot is quite fast and simple – when you don’t encounter problems as you usually do – but what if you need to connect another manufacture’s system. In my case I had Confluence as one data source and luckily there is ready made connector for that data source in Preview state.
Get Confluence and API token
First you need to create Atlassian Confluence Cloud trial or get admin access to you Confluence Cloud or Data Center. Login to your Confluence and create personal access token in https://id.atlassian.com/manage-profile/security/api-tokens.
You should create content to your Confluence and publish it. Create 5-10 pages with some AI generated content which you are sure that you can query then in Copilot bot. I created story about dragon calle Ahti that lives in Turku. This is easy then test in Copilot Studio bot asking “is there dragos in Finland” etc.
Configure Confluence Connector
Then go to your Microsoft tenant Search & intelligence – Microsoft 365 admin center and click Add Connection. Choose Confluence Cloud and click Next (please notice that there is also connector for On-Prem Confluence as well).
Give data and setup authentication. Proper authentication would be OAuth2 but in this demo I just used my account and API token I created in earlier step. If you are build a demo, click Custom setup to get the other tabs visible.
When creating demo and not having same accounts synced from the same place i.e. Azure Entra Id, it will be needed to setup Confluence data to be visible to everyone. In real cases you should keep the access linking and not give everyone permission to see all the Confluence data.
Finish the connector Wizard and you see your connector created. I created two since the ConfluenceCloud1 is using linked permissions and ConfluenceCloud2 has Everyone permission set. This setting cannot be changed afterwards – this is why I have two connectors created.
Copilto Studio bot and knowledge source
Now you are ready to create your Copilot Studio bot. Once you have created it, go to knowledge sources and Add knowledge, select Confluence and then the connector you created.
Now you can test your bot in Copilot Studio and confirm that it works. If you just created the connector it takes some time to crawl the content so be patient and maybe try after couple hours.
Then you should publish your bot.
First go to Channels tab and select Teams. Select Turn on and wait for the below picture to see. Then you need to publish your Copilot (close Teams window, top right corner is Publish button).
Once you have published the bot, come back Teams settings (above picture) and click Open copilot. This takes you to Teams and you can add it to your Teams. If it does not work for first time, try again in 15 minutes.
Cut down other AI hallucination from your bot
If you want to be sure that the answers are only from your knowledge source, you need to change two settings.
Ensure that you have Classic setting on in your Copilot, you can find this in settings
Above setting is not enough, your Copilot could still make up answers. You need to go to Topic tab, then select System and there Conversational boosting
Change the settings for Data sources, select Search only selected sources, select Confluence connector ensure that Classic data selection is off.
And publish your bot again.
Now you can be sure, that all answers are coming from your knowledge source and nowhere else.
If you are interested on how you can set granular permissions and give answer only to the users that has access to Confluence page, please check the next post.