Merge pull request #4515 from jreback/partial_setting

ENH/API:  GH2578, allow ix and friends to partially set when the key is not contained in the object

Author: jreback

doc/source/indexing.rst

@@ -412,6 +412,41 @@ Pandas will detect this and raise ``IndexError``, rather than return an empty st
     >>> df.iloc[:,3:6]
     IndexError: out-of-bounds on slice (end)
 
+.. _indexing.basics.partial_setting:
+
+Setting With Enlargement
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. versionadded:: 0.13
+
+The ``.loc/.ix/[]`` operations can perform enlargement when setting a non-existant key for that axis.
+
+In the ``Series`` case this is effectively an appending operation
+
+.. ipython:: python
+
+   se = Series([1,2,3])
+   se
+   se[5] = 5.
+   se
+
+A ``DataFrame`` can be enlarged on either axis via ``.loc``
+
+.. ipython:: python
+
+   dfi = DataFrame(np.arange(6).reshape(3,2),
+                   columns=['A','B'])
+   dfi
+   dfi.loc[:,'C'] = dfi.loc[:,'A']
+   dfi
+
+This is like an ``append`` operation on the ``DataFrame``.
+
+.. ipython:: python
+
+   dfi.loc[3] = 5
+   dfi
+
 .. _indexing.basics.get_value:
 
 Fast scalar value getting and setting
@@ -431,15 +466,20 @@ Similary to ``loc``, ``at`` provides **label** based scalar lookups, while, ``ia
    df.at[dates[5], 'A']
    df.iat[3, 0]
 
-You can also set using these same indexers. These have the additional
-capability of enlarging an object. This method *always* returns a reference to
-the object it modified, which in the case of enlargement, will be a **new object**:
+You can also set using these same indexers.
 
 .. ipython:: python
 
    df.at[dates[5], 'E'] = 7
    df.iat[3, 0] = 7
 
+``at`` may enlarge the object in-place as above if the indexer is missing.
+
+.. ipython:: python
+
+   df.at[6, 0] = 7
+   df
+
 Boolean indexing
 ~~~~~~~~~~~~~~~~
 

doc/source/release.rst

@@ -92,6 +92,9 @@ pandas 0.13
     an alias of iteritems used to get around ``2to3``'s changes).
     (:issue:`4384`, :issue:`4375`, :issue:`4372`)
   - ``Series.get`` with negative indexers now returns the same as ``[]`` (:issue:`4390`)
+  - allow ``ix/loc`` for Series/DataFrame/Panel to set on any axis even when the single-key is not currently contained in
+    the index for that axis (:issue:`2578`)
+  - ``at`` now will enlarge the object inplace (and return the same) (:issue:`2578`)
 
   - ``HDFStore``
 
@@ -296,7 +299,7 @@ See :ref:`Internal Refactoring<whatsnew_0130.refactoring>`
   - ``tslib.get_period_field()`` and ``tslib.get_period_field_arr()`` now raise
     if code argument out of range (:issue:`4519`, :issue:`4520`)
   - Fix boolean indexing on an empty series loses index names (:issue:`4235`),
-  infer_dtype works with empty arrays.
+    infer_dtype works with empty arrays.
   - Fix reindexing with multiple axes; if an axes match was not replacing the current axes, leading
     to a possible lazay frequency inference issue (:issue:`3317`)
   - Fixed issue where ``DataFrame.apply`` was reraising exceptions incorrectly

doc/source/v0.13.0.txt

@@ -134,6 +134,54 @@ API changes
         df1 and df2
         s1 and s2
 
+Indexing API Changes
+~~~~~~~~~~~~~~~~~~~~
+
+    Prior to 0.13, it was impossible to use an indexer (``.loc/.iloc/.ix``) to set a value that
+    was not contained in the index of a particular axis. (:issue:`2578`). See more at :ref:`here<indexing.basics.partial_setting>`
+
+    In the ``Series`` case this is effectively an appending operation
+
+    .. ipython:: python
+
+       s = Series([1,2,3])
+       s
+       s[5] = 5.
+       s
+
+    .. ipython:: python
+
+       dfi = DataFrame(np.arange(6).reshape(3,2),
+                       columns=['A','B'])
+       dfi
+
+    This would previously ``KeyError``
+
+    .. ipython:: python
+
+       dfi.loc[:,'C'] = dfi.loc[:,'A']
+       dfi
+
+    This is like an ``append`` operation.
+
+    .. ipython:: python
+
+       dfi.loc[3] = 5
+       dfi
+
+    A Panel setting operation on an arbitrary axis aligns the input to the Panel
+
+    .. ipython:: python
+
+       p = pd.Panel(np.arange(16).reshape(2,4,2),
+                   items=['Item1','Item2'],
+                   major_axis=pd.date_range('2001/1/12',periods=4),
+                   minor_axis=['A','B'],dtype='float64')
+       p
+       p.loc[:,:,'C'] = Series([30,32],index=p.items)
+       p
+       p.loc[:,:,'C']
+
 Enhancements
 ~~~~~~~~~~~~
 

pandas/core/frame.py

@@ -1731,18 +1731,12 @@ def set_value(self, index, col, value):
             engine.set_value(series.values, index, value)
             return self
         except KeyError:
-            new_index, new_columns = self._expand_axes((index, col))
-            result = self.reindex(index=new_index, columns=new_columns,
-                                  copy=False)
-            likely_dtype, value = _infer_dtype_from_scalar(value)
 
-            made_bigger = not np.array_equal(new_columns, self.columns)
+            # set using a non-recursive method & reset the cache
+            self.loc[index,col] = value
+            self._item_cache.pop(col,None)
 
-            # how to make this logic simpler?
-            if made_bigger:
-                com._possibly_cast_item(result, col, likely_dtype)
-
-            return result.set_value(index, col, value)
+            return self
 
     def irow(self, i, copy=False):
         return self._ixs(i, axis=0)