Skip to content

Commit 4c4e7ed

Browse files
passrenPeng Ren
andauthored
0.2.2 (#4)
* Update README * Support nested query * Update dependency * Add more time for timeout * Nomalize bracket for array in the query * Add subquery support * Clean up the code * Fix pagnation issue * Fix pagination issue * Refactor builder class * Remove subquery support * Remove unused logging * Superset implementation with QueryDB --------- Co-authored-by: Peng Ren <ia250@cummins.com>
1 parent a169d03 commit 4c4e7ed

27 files changed

+2176
-432
lines changed

README.md

Lines changed: 25 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
[![Python Version](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/downloads/)
99
[![MongoDB](https://img.shields.io/badge/MongoDB-7.0+-green.svg)](https://www.mongodb.com/)
1010
[![SQLAlchemy](https://img.shields.io/badge/SQLAlchemy-1.4+_2.0+-darkgreen.svg)](https://www.sqlalchemy.org/)
11+
[![Superset](https://img.shields.io/badge/Apache_Superset-1.0+-blue.svg)](https://superset.apache.org/docs/6.0.0/configuration/databases)
1112

1213
PyMongoSQL is a Python [DB API 2.0 (PEP 249)](https://www.python.org/dev/peps/pep-0249/) client for [MongoDB](https://www.mongodb.com/). It provides a familiar SQL interface to MongoDB, allowing developers to use SQL to interact with MongoDB collections.
1314

@@ -40,6 +41,9 @@ PyMongoSQL implements the DB API 2.0 interfaces to provide SQL-like access to Mo
4041
- **ANTLR4** (SQL Parser Runtime)
4142
- antlr4-python3-runtime >= 4.13.0
4243

44+
- **JMESPath** (JSON/Dict Path Query)
45+
- jmespath >= 1.0.0
46+
4347
### Optional Dependencies
4448

4549
- **SQLAlchemy** (for ORM/Core support)
@@ -136,37 +140,41 @@ while users:
136140
### SELECT Statements
137141
- Field selection: `SELECT name, age FROM users`
138142
- Wildcards: `SELECT * FROM products`
143+
- **Nested fields**: `SELECT profile.name, profile.age FROM users`
144+
- **Array access**: `SELECT items[0], items[1].name FROM orders`
139145

140146
### WHERE Clauses
141147
- Equality: `WHERE name = 'John'`
142148
- Comparisons: `WHERE age > 25`, `WHERE price <= 100.0`
143149
- Logical operators: `WHERE age > 18 AND status = 'active'`
150+
- **Nested field filtering**: `WHERE profile.status = 'active'`
151+
- **Array filtering**: `WHERE items[0].price > 100`
152+
153+
### Nested Field Support
154+
- **Single-level**: `profile.name`, `settings.theme`
155+
- **Multi-level**: `account.profile.name`, `config.database.host`
156+
- **Array access**: `items[0].name`, `orders[1].total`
157+
- **Complex queries**: `WHERE customer.profile.age > 18 AND orders[0].status = 'paid'`
158+
159+
> **Note**: Avoid SQL reserved words (`user`, `data`, `value`, `count`, etc.) as unquoted field names. Use alternatives or bracket notation for arrays.
144160
145161
### Sorting and Limiting
146162
- ORDER BY: `ORDER BY name ASC, age DESC`
147163
- LIMIT: `LIMIT 10`
148164
- Combined: `ORDER BY created_at DESC LIMIT 5`
149165

150-
## Connection Options
166+
## Limitations & Roadmap
151167

152-
```python
153-
from pymongosql.connection import Connection
168+
**Note**: Currently PyMongoSQL focuses on Data Query Language (DQL) operations. The following SQL features are **not yet supported** but are planned for future releases:
154169

155-
# Basic connection
156-
conn = Connection(host="localhost", port=27017, database="mydb")
170+
- **DML Operations** (Data Manipulation Language)
171+
- `INSERT`, `UPDATE`, `DELETE`
172+
- **DDL Operations** (Data Definition Language)
173+
- `CREATE TABLE/COLLECTION`, `DROP TABLE/COLLECTION`
174+
- `CREATE INDEX`, `DROP INDEX`
175+
- `LIST TABLES/COLLECTIONS`
157176

158-
# With authentication
159-
conn = Connection(
160-
host="mongodb://user:pass@host:port/db?authSource=admin",
161-
database="mydb"
162-
)
163-
164-
# Connection properties
165-
print(conn.host) # MongoDB connection URL
166-
print(conn.port) # Port number
167-
print(conn.database_name) # Database name
168-
print(conn.is_connected) # Connection status
169-
```
177+
These features are on our development roadmap and contributions are welcome!
170178

171179
## Contributing
172180

pymongosql/__init__.py

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
if TYPE_CHECKING:
77
from .connection import Connection
88

9-
__version__: str = "0.2.1"
9+
__version__: str = "0.2.2"
1010

1111
# Globals https://www.python.org/dev/peps/pep-0249/#globals
1212
apilevel: str = "2.0"
@@ -42,6 +42,26 @@ def connect(*args, **kwargs) -> "Connection":
4242
return Connection(*args, **kwargs)
4343

4444

45+
# Register superset execution strategy for mongodb+superset:// connections
46+
def _register_superset_executor() -> None:
47+
"""Register SupersetExecution strategy for superset mode.
48+
49+
This allows the executor and cursor to be unaware of superset -
50+
the execution strategy is automatically selected based on the connection mode.
51+
"""
52+
try:
53+
from .executor import ExecutionPlanFactory
54+
from .superset_mongodb.executor import SupersetExecution
55+
56+
ExecutionPlanFactory.register_strategy(SupersetExecution())
57+
except ImportError:
58+
# Superset module not available - skip registration
59+
pass
60+
61+
62+
# Auto-register superset executor on module import
63+
_register_superset_executor()
64+
4565
# SQLAlchemy integration (optional)
4666
# For SQLAlchemy functionality, import from pymongosql.sqlalchemy_mongodb:
4767
# from pymongosql.sqlalchemy_mongodb import create_engine_url, create_engine_from_mongodb_uri

pymongosql/common.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,10 +17,12 @@ class BaseCursor(metaclass=ABCMeta):
1717
def __init__(
1818
self,
1919
connection: "Connection",
20+
mode: str = "standard",
2021
**kwargs,
2122
) -> None:
2223
super().__init__()
2324
self._connection = connection
25+
self.mode = mode
2426

2527
@property
2628
def connection(self) -> "Connection":

pymongosql/connection.py

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
from .common import BaseCursor
1313
from .cursor import Cursor
1414
from .error import DatabaseError, NotSupportedError, OperationalError
15+
from .helper import ConnectionHelper
1516

1617
_logger = logging.getLogger(__name__)
1718

@@ -35,9 +36,17 @@ def __init__(
3536
to ensure full compatibility. All parameters are passed through directly
3637
to the underlying MongoClient.
3738
39+
Supports connection string patterns:
40+
- mongodb://host:port/database - Core driver (no subquery support)
41+
- mongodb+superset://host:port/database - Superset driver with subquery support
42+
3843
See PyMongo MongoClient documentation for full parameter details.
3944
https://www.mongodb.com/docs/languages/python/pymongo-driver/current/connect/mongoclient/
4045
"""
46+
# Check if connection string specifies mode
47+
connection_string = host if isinstance(host, str) else None
48+
self._mode, host = ConnectionHelper.parse_connection_string(connection_string)
49+
4150
# Extract commonly used parameters for backward compatibility
4251
self._host = host or "localhost"
4352
self._port = port or 27017
@@ -154,6 +163,11 @@ def database(self) -> Database:
154163
raise OperationalError("No database selected")
155164
return self._database
156165

166+
@property
167+
def mode(self) -> str:
168+
"""Get the specified mode"""
169+
return self._mode
170+
157171
def use_database(self, database_name: str) -> None:
158172
"""Switch to a different database"""
159173
if self._client is None:
@@ -267,6 +281,7 @@ def cursor(self, cursor: Optional[Type[BaseCursor]] = None, **kwargs) -> BaseCur
267281

268282
new_cursor = cursor(
269283
connection=self,
284+
mode=self._mode,
270285
**kwargs,
271286
)
272287
self.cursor_pool.append(new_cursor)

pymongosql/cursor.py

Lines changed: 35 additions & 92 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,11 @@
22
import logging
33
from typing import TYPE_CHECKING, Any, Dict, List, Optional, Sequence, Tuple, TypeVar
44

5-
from pymongo.cursor import Cursor as MongoCursor
6-
from pymongo.errors import PyMongoError
7-
85
from .common import BaseCursor, CursorIterator
96
from .error import DatabaseError, OperationalError, ProgrammingError, SqlSyntaxError
10-
from .result_set import ResultSet
7+
from .executor import ExecutionContext, ExecutionPlanFactory
8+
from .result_set import DictResultSet, ResultSet
119
from .sql.builder import ExecutionPlan
12-
from .sql.parser import SQLParser
1310

1411
if TYPE_CHECKING:
1512
from .connection import Connection
@@ -23,25 +20,25 @@ class Cursor(BaseCursor, CursorIterator):
2320

2421
NO_RESULT_SET = "No result set."
2522

26-
def __init__(self, connection: "Connection", **kwargs) -> None:
23+
def __init__(self, connection: "Connection", mode: str = "standard", **kwargs) -> None:
2724
super().__init__(
2825
connection=connection,
26+
mode=mode,
2927
**kwargs,
3028
)
3129
self._kwargs = kwargs
3230
self._result_set: Optional[ResultSet] = None
3331
self._result_set_class = ResultSet
3432
self._current_execution_plan: Optional[ExecutionPlan] = None
35-
self._mongo_cursor: Optional[MongoCursor] = None
3633
self._is_closed = False
3734

3835
@property
3936
def result_set(self) -> Optional[ResultSet]:
4037
return self._result_set
4138

4239
@result_set.setter
43-
def result_set(self, val: ResultSet) -> None:
44-
self._result_set = val
40+
def result_set(self, rs: ResultSet) -> None:
41+
self._result_set = rs
4542

4643
@property
4744
def has_result_set(self) -> bool:
@@ -52,8 +49,8 @@ def result_set_class(self) -> Optional[type]:
5249
return self._result_set_class
5350

5451
@result_set_class.setter
55-
def result_set_class(self, val: type) -> None:
56-
self._result_set_class = val
52+
def result_set_class(self, rs_cls: type) -> None:
53+
self._result_set_class = rs_cls
5754

5855
@property
5956
def rowcount(self) -> int:
@@ -78,74 +75,6 @@ def _check_closed(self) -> None:
7875
if self._is_closed:
7976
raise ProgrammingError("Cursor is closed")
8077

81-
def _parse_sql(self, sql: str) -> ExecutionPlan:
82-
"""Parse SQL statement and return ExecutionPlan"""
83-
try:
84-
parser = SQLParser(sql)
85-
execution_plan = parser.get_execution_plan()
86-
87-
if not execution_plan.validate():
88-
raise SqlSyntaxError("Generated query plan is invalid")
89-
90-
return execution_plan
91-
92-
except SqlSyntaxError:
93-
raise
94-
except Exception as e:
95-
_logger.error(f"SQL parsing failed: {e}")
96-
raise SqlSyntaxError(f"Failed to parse SQL: {e}")
97-
98-
def _execute_execution_plan(self, execution_plan: ExecutionPlan) -> None:
99-
"""Execute an ExecutionPlan against MongoDB using db.command"""
100-
try:
101-
# Get database
102-
if not execution_plan.collection:
103-
raise ProgrammingError("No collection specified in query")
104-
105-
db = self.connection.database
106-
107-
# Build MongoDB find command
108-
find_command = {"find": execution_plan.collection, "filter": execution_plan.filter_stage or {}}
109-
110-
# Apply projection if specified (already in MongoDB format)
111-
if execution_plan.projection_stage:
112-
find_command["projection"] = execution_plan.projection_stage
113-
114-
# Apply sort if specified
115-
if execution_plan.sort_stage:
116-
sort_spec = {}
117-
for sort_dict in execution_plan.sort_stage:
118-
for field, direction in sort_dict.items():
119-
sort_spec[field] = direction
120-
find_command["sort"] = sort_spec
121-
122-
# Apply skip if specified
123-
if execution_plan.skip_stage:
124-
find_command["skip"] = execution_plan.skip_stage
125-
126-
# Apply limit if specified
127-
if execution_plan.limit_stage:
128-
find_command["limit"] = execution_plan.limit_stage
129-
130-
_logger.debug(f"Executing MongoDB command: {find_command}")
131-
132-
# Execute find command directly
133-
result = db.command(find_command)
134-
135-
# Create result set from command result
136-
self._result_set = self._result_set_class(
137-
command_result=result, execution_plan=execution_plan, **self._kwargs
138-
)
139-
140-
_logger.info(f"Query executed successfully on collection '{execution_plan.collection}'")
141-
142-
except PyMongoError as e:
143-
_logger.error(f"MongoDB command execution failed: {e}")
144-
raise DatabaseError(f"Command execution failed: {e}")
145-
except Exception as e:
146-
_logger.error(f"Unexpected error during command execution: {e}")
147-
raise OperationalError(f"Command execution error: {e}")
148-
14978
def execute(self: _T, operation: str, parameters: Optional[Dict[str, Any]] = None) -> _T:
15079
"""Execute a SQL statement
15180
@@ -162,11 +91,25 @@ def execute(self: _T, operation: str, parameters: Optional[Dict[str, Any]] = Non
16291
_logger.warning("Parameter substitution not yet implemented, ignoring parameters")
16392

16493
try:
165-
# Parse SQL to ExecutionPlan
166-
self._current_execution_plan = self._parse_sql(operation)
94+
# Create execution context
95+
context = ExecutionContext(operation, self.mode)
96+
97+
# Get appropriate execution strategy
98+
strategy = ExecutionPlanFactory.get_strategy(context)
99+
100+
# Execute using selected strategy (Standard or Subquery)
101+
result = strategy.execute(context, self.connection)
167102

168-
# Execute the execution plan
169-
self._execute_execution_plan(self._current_execution_plan)
103+
# Store execution plan for reference
104+
self._current_execution_plan = strategy.execution_plan
105+
106+
# Create result set from command result
107+
self._result_set = self._result_set_class(
108+
command_result=result,
109+
execution_plan=self._current_execution_plan,
110+
database=self.connection.database,
111+
**self._kwargs,
112+
)
170113

171114
return self
172115

@@ -236,15 +179,6 @@ def fetchall(self) -> List[Sequence[Any]]:
236179
def close(self) -> None:
237180
"""Close the cursor and free resources"""
238181
try:
239-
if self._mongo_cursor:
240-
# Close MongoDB cursor
241-
try:
242-
self._mongo_cursor.close()
243-
except Exception as e:
244-
_logger.warning(f"Error closing MongoDB cursor: {e}")
245-
finally:
246-
self._mongo_cursor = None
247-
248182
if self._result_set:
249183
# Close result set
250184
try:
@@ -274,3 +208,12 @@ def __del__(self):
274208
self.close()
275209
except Exception:
276210
pass # Ignore errors during cleanup
211+
212+
213+
class DictCursor(Cursor):
214+
"""Cursor that returns results as dictionaries instead of tuples/sequences"""
215+
216+
def __init__(self, connection: "Connection", **kwargs) -> None:
217+
super().__init__(connection=connection, **kwargs)
218+
# Override result set class to use DictResultSet
219+
self._result_set_class = DictResultSet

0 commit comments

Comments
 (0)