Add support for JSON Index creation syntax in SQL Server 2025 (TSql170) #147

Copilot · 2025-07-22T00:10:44Z

This PR implements comprehensive support for JSON Index creation syntax in SQL Server 2025 (TSql170 parser) as requested in the original issue.

Syntax Supported

CREATE JSON INDEX name ON table_name (json_column_name)
  [ FOR ( sql_json_path [ , ...n ] ) ]
  [ WITH ( json_index_option [ , ...n ] ) ]

Where json_index_option includes:

FILLFACTOR = fillfactor
DROP_EXISTING = { ON | OFF }
ONLINE = OFF
ALLOW_ROW_LOCKS = { ON | OFF }
ALLOW_PAGE_LOCKS = { ON | OFF }
MAXDOP = max_degree_of_parallelism
DATA_COMPRESSION = { NONE | ROW | PAGE }

Implementation Details

AST Support

Added CreateJsonIndexStatement class to Ast.xml with properties:
- Name - Index name
- OnName - Table name
- JsonColumn - JSON column reference
- JsonPaths - Collection of JSON path strings for FOR clause
- IndexOptions - Collection of index options for WITH clause
- OnFileGroupOrPartitionScheme - Filegroup specification for ON clause

Parser Support

Added createJsonIndexStatement grammar rule to TSql170.g
Implemented proper keyword recognition using NextTokenMatches(CodeGenerationSupporter.Json)
Integrated with existing CREATE statement parsing flow
Supports optional FOR, WITH, and ON clauses with proper validation

Script Generation

Created SqlScriptGeneratorVisitor.CreateJsonIndexStatement.cs
Generates properly formatted CREATE JSON INDEX statements
Handles all syntax elements with appropriate spacing and line breaks

Test Coverage

Added comprehensive test suite covering:

Basic JSON index creation
JSON indexes with single and multiple JSON paths
JSON indexes with various WITH options
JSON indexes with filegroup specifications
Schema-qualified table names and quoted identifiers
Complex combinations of all features

Example test cases:

-- Basic
CREATE JSON INDEX IX_JSON_Basic ON dbo.Users (JsonData);

-- With JSON paths
CREATE JSON INDEX IX_JSON_Paths ON dbo.Users (JsonData)
    FOR ('$.name', '$.email', '$.profile.age');

-- With options and filegroup
CREATE JSON INDEX IX_JSON_Full ON dbo.Users (JsonData)
    FOR ('$.orders[*].amount', '$.orders[*].date')
    WITH (DROP_EXISTING = OFF, ALLOW_ROW_LOCKS = ON)
    ON [JsonIndexes];

Validation

All tests pass across all SQL Server versions:

✅ TSql170 (SQL Server 2025): Successfully parses and generates JSON indexes
✅ TSql160-TSql80: Correctly rejects JSON index syntax with expected parse errors
✅ Maintains backward compatibility with existing functionality

The implementation correctly handles version-specific behavior where JSON indexes are only supported in SQL Server 2025 and later.

Fixes #146.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

central.sonatype.com
- Triggering command: curl -L -o antlr/antlr-2.7.5.jar REDACTED (dns block)
www.antlr2.org
- Triggering command: /usr/lib/dotnet/dotnet /usr/lib/dotnet/sdk/8.0.117/MSBuild.dll /nologo /nodemode:1 /nodeReuse:true /low:false (dns block)
- Triggering command: wget -O antlr/antlr-2.7.5.jar REDACTED (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to my firewall allow list

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: llali <17187348+llali@users.noreply.github.com>

…eneration Co-authored-by: llali <17187348+llali@users.noreply.github.com>

…orrect baseline Co-authored-by: llali <17187348+llali@users.noreply.github.com>

…sions Co-authored-by: llali <17187348+llali@users.noreply.github.com>

llali

Don't add antlr-2.7.5.jar to this PR

llali · 2025-07-22T14:59:26Z

remove the tmp folder from this PR

Co-authored-by: llali <17187348+llali@users.noreply.github.com>

Copilot · 2025-07-22T15:02:54Z

Don't add antlr-2.7.5.jar to this PR

Removed antlr-2.7.5.jar from the PR. The file should be downloaded automatically by the build system as configured in Directory.Build.props. (commit 73a1645)

Co-authored-by: llali <17187348+llali@users.noreply.github.com>

Copilot · 2025-07-22T15:06:41Z

remove the tmp folder from this PR

Removed the tmp folder from the PR and added tmp/ to .gitignore to prevent future commits of build artifacts. (commit 9de9b7f)

global.json

Co-authored-by: llali <17187348+llali@users.noreply.github.com>

SqlScriptDom/NUL

Co-authored-by: llali <17187348+llali@users.noreply.github.com>

Test/SqlDom/Baselines170/JsonIndexTests170.sql

.gitignore

zijchen · 2025-07-22T19:51:40Z

Test/SqlDom/ParserErrorsTests.cs

+        public void CreateJsonIndexStatementErrorTest()
+        {
+            // JSON Index syntax should not be supported in SQL Server versions prior to 2025 (TSql170)
+            // Test basic JSON Index syntax in older versions


Remove these, all tests against older parser versions are unnecessary here, they should be covered in Only170SyntaxTests already

Removed the ErrorTest160 calls from ParserErrorsTests.cs as they are unnecessary - older version rejection is already covered in Only170SyntaxTests. (commit b6073da)

SqlScriptDom/Parser/TSql/Ast.xml

…y TSql170 tests needed Co-authored-by: zijchen <13544267+zijchen@users.noreply.github.com>

Copilot · 2025-07-22T20:00:25Z

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

www.antlr2.org
- Triggering command: dotnet test Test/SqlDom/UTSqlScriptDom.csproj --filter DisplayName~Json --no-restore -v q (dns block)
- Triggering command: /usr/lib/dotnet/dotnet /usr/lib/dotnet/sdk/8.0.117/MSBuild.dll /nologo /nodemode:1 /nodeReuse:false /low:false (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to my firewall allow list

llali · 2025-07-28T16:53:06Z

SqlScriptDom/Parser/TSql/Ast.xml

+    <Member Name="JsonColumn" Type="Identifier" Summary="The JSON column for the index."/>
+    <Member Name="ForJsonPaths" Type="StringLiteral" Collection="true" Summary="The JSON paths specified in the FOR clause. Optional may have zero elements."/>
+    <InheritedMember Name="IndexOptions" ContainerClass="IndexStatement" />
+    <Member Name="OnFileGroupOrPartitionScheme" Type="FileGroupOrPartitionScheme" Summary="The filegroup or partition scheme. Might be null."/>


looks like file group is not supported for json index. remove file group from ast.xml and from TSQL170.g and apply the same change in tests

Removed filegroup support from JSON Index syntax as requested. The changes include:

Removed OnFileGroupOrPartitionScheme member from AST

Removed filegroup handling from TSql170.g grammar rule

Updated script generator to exclude filegroup generation

Removed filegroup test cases from test scripts and baselines

JSON indexes now support only the core syntax without ON filegroup clause. (commit eee2f48)

Co-authored-by: llali <17187348+llali@users.noreply.github.com>

Copilot · 2025-07-28T17:03:01Z

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

www.antlr2.org
- Triggering command: dotnet build SqlScriptDom/Microsoft.SqlServer.TransactSql.ScriptDom.csproj --no-restore -v q (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

llali · 2025-07-28T19:34:10Z

Test/SqlDom/Only170SyntaxTests.cs

        private static readonly ParserTest[] Only170TestInfos =
        {
            new ParserTest170("RegexpTVFTests170.sql", nErrors80: 1, nErrors90: 1, nErrors100: 0, nErrors110: 0, nErrors120: 0, nErrors130: 0, nErrors140: 0, nErrors150: 0, nErrors160: 0),
+            new ParserTest170("JsonIndexTests170.sql", nErrors80: 2, nErrors90: 10, nErrors100: 10, nErrors110: 10, nErrors120: 10, nErrors130: 10, nErrors140: 10, nErrors150: 10, nErrors160: 10),


fix the number of errors after your last commit. it should be 8 errors now not 10

Fixed the error count from 10 to 8 errors for JSON Index tests. After removing filegroup support, there are now 8 CREATE JSON INDEX statements in the test file instead of 10, so the expected error count for older parser versions was updated accordingly. (commit 04e7f67)

…s after filegroup removal Co-authored-by: llali <17187348+llali@users.noreply.github.com>

Copilot · 2025-07-28T19:39:28Z

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

www.antlr2.org
- Triggering command: dotnet build --no-restore --verbosity quiet (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Initial plan

269592c

Copilot AI assigned Copilot and llali Jul 22, 2025

Copilot started work on behalf of llali July 22, 2025 00:10 View session

Copilot AI and others added 4 commits July 22, 2025 00:17

Initial planning and exploration for JSON Index support

90ad82c

Co-authored-by: llali <17187348+llali@users.noreply.github.com>

Add JSON Index support - AST definitions, grammar rules, and script g…

3e23f8b

…eneration Co-authored-by: llali <17187348+llali@users.noreply.github.com>

Complete JSON Index implementation - fix parser recognition and add c…

9d7e0db

…orrect baseline Co-authored-by: llali <17187348+llali@users.noreply.github.com>

Final fix - correct error count expectations for older SQL parser ver…

753a74b

…sions Co-authored-by: llali <17187348+llali@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add support for Json Index~~ Add support for JSON Index creation syntax in SQL Server 2025 (TSql170) Jul 22, 2025

Copilot AI requested a review from llali July 22, 2025 00:44

Copilot finished work on behalf of llali July 22, 2025 00:44

llali reviewed Jul 22, 2025

View reviewed changes

Copilot started work on behalf of llali July 22, 2025 14:58 View session

Remove antlr-2.7.5.jar file from repository

73a1645

Co-authored-by: llali <17187348+llali@users.noreply.github.com>

Copilot finished work on behalf of llali July 22, 2025 15:03

Copilot AI requested a review from llali July 22, 2025 15:03

Copilot started work on behalf of llali July 22, 2025 15:03 View session

Remove tmp folder from repository and add to .gitignore

9de9b7f

Co-authored-by: llali <17187348+llali@users.noreply.github.com>

Copilot finished work on behalf of llali July 22, 2025 15:07

llali reviewed Jul 22, 2025

View reviewed changes

global.json Show resolved Hide resolved

Copilot started work on behalf of llali July 22, 2025 15:21 View session

Revert global.json SDK version back to 8.0.410

4865dd4

Co-authored-by: llali <17187348+llali@users.noreply.github.com>

llali reviewed Jul 22, 2025

View reviewed changes

SqlScriptDom/NUL Outdated Show resolved Hide resolved

Copilot AI requested a review from llali July 22, 2025 15:24

Copilot finished work on behalf of llali July 22, 2025 15:24

Copilot started work on behalf of llali July 22, 2025 15:24 View session

Remove NUL file - ANTLR build artifact

373dc4d

Co-authored-by: llali <17187348+llali@users.noreply.github.com>

ssreerama reviewed Jul 22, 2025

View reviewed changes

Test/SqlDom/Baselines170/JsonIndexTests170.sql Outdated Show resolved Hide resolved

ssreerama reviewed Jul 22, 2025

View reviewed changes

.gitignore Show resolved Hide resolved

Copilot AI requested a review from ssreerama July 22, 2025 19:26

Copilot finished work on behalf of ssreerama July 22, 2025 19:26

Copilot started work on behalf of ssreerama July 22, 2025 19:26 View session

Copilot finished work on behalf of ssreerama July 22, 2025 19:31

zijchen reviewed Jul 22, 2025

View reviewed changes

Copilot started work on behalf of zijchen July 22, 2025 19:53 View session

Remove ErrorTest160 calls from ParserErrorsTests for JSON Index - onl…

b6073da

…y TSql170 tests needed Co-authored-by: zijchen <13544267+zijchen@users.noreply.github.com>

Copilot AI requested a review from zijchen July 22, 2025 20:00

Copilot finished work on behalf of zijchen July 22, 2025 20:00

fixing the test

ab3a74f

llali approved these changes Jul 23, 2025

View reviewed changes

llali reviewed Jul 28, 2025

View reviewed changes

Copilot started work on behalf of llali July 28, 2025 16:53 View session

Remove filegroup support from JSON Index syntax

eee2f48

Co-authored-by: llali <17187348+llali@users.noreply.github.com>

Copilot finished work on behalf of llali July 28, 2025 17:03

Copilot AI requested a review from llali July 28, 2025 17:03

llali reviewed Jul 28, 2025

View reviewed changes

Copilot started work on behalf of llali July 28, 2025 19:34 View session

Fix JSON Index error count in Only170SyntaxTests - correct to 8 error…

04e7f67

…s after filegroup removal Co-authored-by: llali <17187348+llali@users.noreply.github.com>

Copilot AI requested a review from llali July 28, 2025 19:39

Copilot finished work on behalf of llali July 28, 2025 19:39

llali approved these changes Jul 28, 2025

View reviewed changes

ssreerama approved these changes Jul 29, 2025

View reviewed changes

zijchen approved these changes Jul 30, 2025

View reviewed changes

llali merged commit 0a26b6d into main Jul 30, 2025
5 checks passed

Add support for JSON Index creation syntax in SQL Server 2025 (TSql170) #147

Add support for JSON Index creation syntax in SQL Server 2025 (TSql170) #147

Uh oh!

Conversation

Copilot AI commented Jul 22, 2025 • edited by llali Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Syntax Supported

Implementation Details

AST Support

Parser Support

Script Generation

Test Coverage

Validation

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

llali left a comment

Choose a reason for hiding this comment

Uh oh!

llali commented Jul 22, 2025

Uh oh!

Copilot AI commented Jul 22, 2025

Uh oh!

Copilot AI commented Jul 22, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zijchen Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI commented Jul 22, 2025

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

llali Jul 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI commented Jul 28, 2025

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

llali Jul 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI commented Jul 28, 2025

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Copilot AI commented Jul 22, 2025 •

edited by llali

Loading